How can I use free transform to rotate an picture while keeping resolution

On that application, you can not.

Use a vector based program like Corel Draw or Illustrator.

If you stretch, squish and squash a photo inside them the file’s size will be constant. With embedded and with linked images as well (in programs like InDesign)

When you increase the size of the image, the image’s size will remain intact, but the catch is that you are actually squishing and squashing the print resolution of the file in an inverse proportion.

If you increase the image’s size by 2, you reduce the print resolution by 2. There is no way around. It is just the way things work.

You need to start with a good image size from the start that will be able to withstand your manipulation.

Some free applications that you can use are Inkscape, Scribus and Gravit designer.