... would it be okay to let the presentation layer create the container and register the required dependencies?
No.
The Presentation Layer is responsible for displaying system information to the user.
The Infrastructure Layer is responsible for providing technical resources that will support the upper layers. That is, the latter should have this kind of concern.
Some problems you may have when registering the container and its dependencies in the presentation layer:
As you are using an architecture following Domain Driven Design (DDD), you will probably have a Business Layer, where "services" will depend on queries to repositories.
In this scenario, depending on your implementation, your business project (called Domain in DDD) would depend directly on your Layer project, which is inconsistent with the DDD approach.
Another scenario that you may have problem: If your project's Layer Layer changes in the future to Asp.Net MVC, Web Form, etc ... you will have to keep changing / moving this code that resolves / registers the dependencies, that is, you will have more maintenance and fiddling with this code again: - (.
Register the dependencies in the Infrastructure Layer.
... Or should ALL of these dependencies be registered to the Infrastructure layer?
Yes, for common dependencies to projects / layers, especially Repositories and Services.
... Or, both? For example: does the presentation layer register things like Views and does the Infrastructure layer register the dependencies that are common to all layers?
How to register / resolve dependencies is not the responsibility of the Presentation Layer, I advise you to register the dependencies in the Infrastructure Layer, so you have a single point and do not need to be servicing in two places.