If you install with ansible it should be included by default. You can check if your nginx config contains the cache lines. There is also a line you can uncomment to see the cache status with each request.
https://github.com/LemmyNet/lemmy-ansible/blob/main/templates/nginx.conf#L71
If there are still problems you should open a new issue. We cant leave issues open forever because they go stale and dont account for new features. By the way we are planning to implement multi-communities.