My theory follows thus:
Caffrey has been manipulated from the start. Burke planned for him to go after Kate, and knowing him, would be easily found again afterwards. Caffrey knew he'd get sent back and have to repeat his sentence, and therefore, look for any loophole to get out of it. In effect, he lands himself into the custody of the FBI. Burke knows that as long as Neil thinks Kate is in New York, he'll stay there and continue to assist the FBI with cases using his skills. Plus, he'll keep for him for an added 4 yeras rather than the few months before Caffrey would've been released. Keeping him in Burke's sight keeps Neil on the straight and narrow (or rather, the less crooked and wide in Caffrey's case). An added bonus is that he'll do anything for Kate which Burke could use later to his advantage. A tad convoluted, I realize, however, Burke is the man that caught Caffrey after all. He has to be somewhat clever, eh? Thoughts? An interesting theory, or am I giving Peter too much credit? I smell conspiracy!!