There’s a lot one side of the market can do with collusion to drive up prices.

For example, the housing supply available on the market right now often fluctuates over time - once someone has a long term tenancy, their price is locked in until the next legal opportunity to change the price. If there is a low season - fewer people living in an area, higher vacancy rate - in a competitive market, tenants often want a long term contract (if they are planning on staying), and the landlord who can offer that will win the contract. The landlord gets income during the low period, but forgo a higher rent they might get during a high period. Another landlord who tries to take a hardline policy of insisting tenants renew their contract during the high period would lose out - they would just miss out on rent entirely during the low period, likely making less than the other landlord.

Now if the landlords form a cartel during the low period, and it is not possible to lease a rental property long term, then the tenants have no choice but to be in a position to re-negotiate price during the high period. Landlords avoid having to choose between no revenue during the low period and higher during the high period, or a consistent lower rent - instead they get the lower rent during the low period (at a slightly lower occupancy rate, but shared across all the landlords) AND the higher rent during the high period.