Your final conclusion is the best explanation for VB.Net to exist. I will not find anything easily, but there is some scattered information that shows this to be it. Unlike some languages that are better at some kind of problem, when comparing these two it is difficult to point out clearly why one is better than another if you do not have any previous background.
Current versions have fewer differences and are almost completely interchangeable.
Performance
In terms of performance there is nothing in the languages that profoundly change the result, mainly because most of the time will be spent executing functions that are the same in both (almost always written in C # and transformed into CIL code ). It makes more difference to the platform where it runs than the language itself. Nor will I talk about the question of which algorithms usually make the most difference.
Of course some language features may make a small difference, but you're probably comparing different things. So the comparison becomes unfair. To get equal results you need to write different codes. But note that the difference will be very small and you can not tell clearly which one is most advantageous.
In some cases the same code can produce better results in one than the other, but with little modification can equate both or even reverse the result. Comparing language performance is very complicated, in practice it does not. There is no faster language than the other in general terms, especially when they have very similar basic philosophies.
Styles
I do not think a list of small differences would help define something relevant to choosing one or the other. What is certain is that VB.Net has a little less ceremony than C #, but very little. There is more convenience in one or the other depending on how you look at the problem.
Generally, types, memory management, almost all basic algorithms and data structure constructs work the same, the difference is essentially syntactic.
Legacy of VB
VB.Net can use a style a bit closer to the original VB but it is not recommended to use it in new code. It may seem a bit easier to use this style but deep down it brings more problems in the long run. As there is this recommendation, in practice the advantage of familiarity is not relevant.
It's an illusion that you'll make an easier transition from VB to VB.Net. Legacy code does not converge so simply and the differences from VB to VB.Net are so great that the work of learning VB.Net may well be availed to learn C #. The different syntax between languages is the easy part of what you have to learn.
Difference in language support
There are a few more examples and material available for C # than VB.Net, but nothing significant that makes much difference.
C # has a feeling of being the official language and VB.Net is there because of the legacy. But this is not proven if the analysis is very objective.
Conclusion
Any criterion that can be observed seems to be too subjective. I doubt if anyone can give another reason to choose one or the other but to taste.