It could be your "transfer switch". The transfer switch detects when power is coming from shore power or not. If it senses no shore power, it will flip over to the 120 AC volt coming from the inverter.
When you plug back into shore power, the transfer switch will flip again, back to 120 v power, from "shore power" and close off power coming from the inverter.
It's possible this switch is failing.
You need to check power the power output at the inverter itself (where the transfer switch is plugged into it), and then check the power at the plug (where the refrigerator is).
__________________
History is not about the past, it's an explanation of the present.
2019 Montana High Country 375FL
2014 Chevy Silverado Duramax, 6.6L Dually