DNS: Split-Brain DNS via Unbound. Inside the local Wi-Fi network, survey.foo.pf correctly resolves to the private IP of the Apache Reverse Proxy.
Interviewer App Version: 26.06
The Problem:
When tablets are connected to the local Wi-Fi (VLAN Y), user authentication and questionnaire synchronization work instantly and perfectly. However, map downloads refuse to progress.
When switching the exact same tablets to a 4G/LTE mobile network (accessing the server through its public IP), maps download perfectly fine (reaching 5% within seconds…).
What we have already checked and ruled out:
Firewall Blocks: Tested with an ALL to ALL pass rule from the tablet’s local IP to any destination. No dropped packets or rejects in the firewall live logs.
Android Private DNS: Completely disabled on the devices to ensure they are strictly using our local Unbound DNS (and effectively hitting the private IP of the proxy).
MTU / Fragmentation Issues: Performed an ICMP ping test from a PC on the same Wi-Fi network to the proxy using ping -f -l 1472. Packets pass flawlessly without fragmentation. Inside the Docker host, the bridge network MTU is also aligned at 1500.
Hardware Offloading: TSO, GSO, GRO, and Hardware Checksum Offloading have been disabled on the OPNsense firewall, the Proxmox network interfaces, and inside the proxy VM.
Apache Configuration:
ProxyPreserveHost On and X-Forwarded-Proto “https” are set.
Attempted to force Protocols http/1.1 to avoid HTTP/2 multiplexing/flow-control saturation over high-speed local Wi-Fi.
Attempted disabling proxy buffering using flushpackets=on enablereuse=on on the ProxyPass directive.
App Cache: Cleared the Interviewer app cache and data, but the behavior remains identical.
Since text-based synchronization (JSON data) works seamlessly over Wi-Fi, the issue seems strictly tied to downloading large binary files locally through the proxy routing into the Docker network layer.
Has anyone encountered a similar behavior with large file streams involving Docker deployments in segmented local networks? Are there specific Docker daemon network tweaks or Apache caching variables we might have missed for this setup?
please be aware that the download progress for large maps may get stuck and the interface stop refreshing in the Interviewer App as per this reported issue. The maps, however, get downloaded eventually.
You wrote:
Please confirm you are not just getting 5% of the map, but the whole map, to confirm success. Also, we have no idea what 5% is: 100kb? 1mb? 10mb? Or, correspondingly, what is the map file size is to judge on what a “large file” is.
If the map is downloaded successfully over 4G and not over WiFi - this seems not to be an application issue, but a networking issue, which would be tightly related to the infrastructure only you can access to experiment with.
That said, I pushed your description to a couple of AI engines and they spit neatly organized recommendations ranging various hypotheses by prevalence and agreement with your described symptoms. I’d recommend you do the same and then follow the recommendations you receive to diagnose the issue. Among some of the mentioned hypotheses were:
TCP window scaling / state handling in OPNsense
Proxy buffering / flow-control deadlock
MTU / PMTUD edge case (less likely but possible)
Android / Wi‑Fi stack interaction
with AI concluding that:
the most likely root cause is: TCP window scaling or flow-control mis-handling in the inter-VLAN path (OPNsense or bridge layer)
Given the degree of network complexity you’ve encountered and the depth of what you’ve already tried, you can also try specialized networking forums, where you may get a more precise diagnostics (if you haven’t done so yet). For example, Proxmox has one here: https://forum.proxmox.com/
PS: perhaps, another advise is to not over-engineer it. Start with something that works, then add bells and whistles to make it better, always maintaining the working state. If you follow this pattern, the last thing you’ve changed is usually the culprit and saves you tons of time on diagnostics and troubleshooting.
Thank you for your reply and for pointing out issue #3951.
After further testing based on your insights, I have some very specific findings to share. We are running version 26.06 (build 40485) on both the Server (Docker) and the Interviewer App (Android).
You are entirely right about the UI freeze: the app’s interface freezes up and requires the “leave it completely untouched” workaround in ALL scenarios (4G, throttled Wi-Fi, and unthrottled Wi-Fi).
However, there is a major difference in how the download actually behaves underneath the freeze:
Over 4G / LTE (Slower speed ~1 MB/s): Even though the UI freezes, the progress bar successfully jumps to 5% within seconds, and the map completely downloads in the background. The final file is fully functional.
Over Local Wi-Fi (High speed ~16 MB/s): The progress bar stays stuck at 0% indefinitely. Even if left untouched for over 30 minutes, the download completely fails, leaving a 0 byte.tpk.part file on the tablet’s storage.
To diagnose this, I ran a curl test from a PC on the same Wi-Fi network. The PC successfully downloaded the 309.5 MB map in 20 seconds at 16.5 MB/s with a perfect 200 OK and correct Content-Length. This completely rules out any MTU, firewall, or network routing drops on our Proxmox/OPNsense infrastructure.
The Solution / Workaround:
The root cause is indeed tied to the app’s I/O logic. On a high-speed local network, the server pushes data much faster than the app can write it to the tablet’s internal storage, causing the background download socket to choke and die at 0%.
To fix this without touching the app, I enabled mod_ratelimit on our Apache reverse proxy strictly for the /api/interviewer/v2/maps/details endpoint, throttling the download speed to 1.5 MB/s (simulating 4G speed).
With this server-side speed limit active over Wi-Fi, the app still freezes, but the progress bar successfully hits 5% after a few seconds, and the 309.5 MB map now completes perfectly in the background!
I hope this case study helps other users facing similar issues in local high-speed enterprise networks.
v26.07 released yesterday should address the issue that I pointed to (with interface freeze). Please see if your issue is also resolved with this update or it remains a problem.