It seems very likely that they were not ready for this realm and will either return at a later time to you or someone near you or they will remain in the afterlife having completed their mission. I do not really have experience with this but I do believe in these theories that other people have also agreed to.
GummyBear, the vicar, being extremely religious, probably had that one belief where it is said that non baptized people go to hell due to them not being released of our ancestors' sin (Adam and Eve), the sin that all people are born with. This sounds really exaggerated though so you should not dwell on his words. Be at peace because your brother is in a lovely place. Or who knows, maybe he's right next to you and you did not notice!

I have always believed that as spirits we chose to come here to learn and choose our path , I believe that when babies are miscarried they have made a last minute change and have decided to wait for another time to return here.