Encryption basically works like this:
- (a) What is written using the private key can only be read with the public key.
- (b) What is written using the public key can only be read with the private key.
The idea is for your public key to share, whereas the private key must be saved and protected in every possible way. If your private key leaks, the entire encryption protection guarantee will be lost.
The RSA (or any other asymmetric key) algorithm assumes that your private key is secure. It's not part of the encryption algorithm to tell how or what you do to save this private key, it just uses the key you give it to. The key generation algorithm also does not say how you should protect your keys, it just produces them and delivers you.
Case (b) above is used for when someone wants to write an encrypted message that only you can read. Once it is created with your public key, then anyone can write it, after all the key is public. However, only you, who own the private key, can read.
Case (a) above is used for digital signature. If you publish an encrypted message with your private key, everyone can read using the public key. However, your authorship will be guaranteed and confirmed, as the only way this message will be readable with your public key is if it has been generated with your private key that only you have access to. This is to ensure the authenticity of information.
If the hacker has access to your private key, he can take control of your encryption, read your private messages, and also pass you by. It's more or less the same thing that happens when some hacker has access to your password.
If you are tempted to distribute your private key within the application, you are probably doing it wrong, because you should never distribute your private key.
If the purpose of the application is to send some message to a central server controlled by you, it will only need the public key used by the server. The private key must be well protected and well stored inside the server and never leave it.
If the application also needs to authenticate the messages generated by it and certify the author ID, you can generate a different public and private key pair for each application installation (within the application itself) and send the public key to you or to a third party. Each application should keep the guard of its own private key as best as possible. And again, you should never put the server's private key inside the application.
Within the application, the protection of the private key can be done by any means that offers a minimum of security, such as put in some internal file. However, you do not have to kill yourself to leave this private key of the ultra-secure application because each installation will use a different private key. So if a hacker can get the private key used in the application installed on the X install, only the X install is compromised, not all installations and neither the server. In this way, your care will only be to ensure that the application does not leak its private key.