When we shipped calls and push notifications in ManOps - our internal team operations product - we had 5,564 automated tests passing (3,298 web and 2,266 Flutter) and two simulator and emulator passes completed. What we could not verify at the time of writing: real-phone FCM and APNs VoIP delivery, lock-screen call ringing, answer-on-one-device cancelling the ring on another, and whether the APNs and FCM environment variables on Vercel actually deliver notifications to real phones. This post documents what we built, what the tests covered, and what remains unverified - because honest limits are more useful than manufactured confidence.
What we built (verified)
ManOps is Zovintra's internal product - a team operations app running as a web app (PWA), with a Flutter native app for iOS and Android built on the same API. The calls and push backend was deployed to production around 8-9 October 2026 [CODE verified in repo `webapp/src/lib/calls/`, `webapp/src/lib/push/`].
The implementation covers:
- Direct and group ringing calls using Jitsi (8x8).
- FCM data messages for Android push - device tokens stored with kind 'fcm'.
- APNs VoIP push (PushKit + CallKit) for iPhone via device tokens with kind 'apns_voip' and an apns_env field ('sandbox' or 'production').
- A 60-second ring window (RING_WINDOW_MS in ring-constants.ts).
- Accept, decline and cancel flows.
- A 'do not disturb' status that suppresses the ring push and creates only an inbox entry.
- Android 'Calls and ringing' guidance screen at /settings/calls.
- Flutter native app with lock-screen call handling code in native/lib/features/calls/.
The native app uses Flutter with a minimum iOS 15.1 (set in Podfile) and minimum Android API 26 (minSdk = 26, forced by Jitsi SDK). CocoaPods is required for iOS because the Jitsi SDK has no Swift Package Manager package.
What the tests covered
The test suite is large. As of the 9 October 2026 state of the repo: 3,298 vitest tests for the web app (3,296 passing, 2 skipped) and 2,266 Flutter tests for the native app (2,261 passing, 5 skipped) [TESTS: run in a clean worktree, 9 Oct 2026].
The calls and push tests specifically include:
- Ring timeout: ring-timeout.test.ts (8 tests) - verifies the 60-second ring window.
- Call push dispatch: call-push.test.ts (18 tests) - verifies correct FCM/APNs payload construction.
- Device push registration: device-push.test.ts (8 tests).
- Push API routes: push-routes.test.ts (17 tests).
- Do-not-disturb: dnd-status.test.ts (5 tests).
- Flutter native: tests in native/test/calls/ covering push parsing, ring-once ledger, 60-second staleness, accept/decline/cancel, registration, guidance screen and the call screen.
Two device passes were completed before this post was written: a simulator and emulator pass for iOS and Android (documented in DEVICE_PASS_IOS.md and DEVICE_PASS_ANDROID.md in the repo). Those passes verified the app UI, navigation and core flows. They were completed before the calls work was merged, so calls were not part of those passes.
What the tests did not cover
Here is the list that matters - the things we documented as not yet verified, directly from the project's current state notes at the time of writing (9 October 2026):
- Real-phone call test. The test script (docs/mobile/DEVICE_TEST_CALLS.md) exists but had not been executed by the owner at the time these notes were written. No real FCM or VoIP ring, lock-screen answer, answer-on-one-device-stops-the-other, DND-on-device or sign-out token removal had been observed on a real phone.
- Production smoke after the 8-9 October deploy for calls and push. Whether the device_tokens table had any rows with real device tokens (the count was 0 as of 3 October).
- APNs and FCM environment variables on Vercel. The Vercel environment variable names are in the code; the values were not inspected. HTTP/2 to APNs from a serverless function was flagged in the project documentation as untested.
- Lock-screen ringing on a real device. The Flutter code for CallKit integration is written and tested in unit tests; CallKit behaviour on a real locked iPhone was not observed.
- Jitsi media (audio and video quality). The Jitsi join flow is tested but the actual call media through 8x8 on real devices in real network conditions was not tested.
- OEM battery optimisation behaviour on Android. Some Android manufacturers aggressively kill background processes; FCM data messages on doze mode or battery-optimised apps may behave differently from the emulator.
This list is not an indictment of the build quality. It reflects the normal boundary between what automated tests can verify and what requires real devices, real network conditions and real users.
Why we shipped with documented limits
ManOps is our internal product, used by our team. The team size is small (five live database users as of October 2026) and the usage pattern is controlled. Shipping with documented untested areas to an internal team is a reasonable trade-off - we get real-world signal we cannot get from tests, and the blast radius of a problem is limited to us.
The key word is documented. The limits were written down in the project's state document before we shipped. The owner knew the exact list of what had not been verified. The real-device call test script exists and is ready to run - it just had not been executed when we wrote these notes.
This is different from shipping without knowing what is untested. Undocumented unknowns are the dangerous kind. A documented list of pending verifications is a queue, not a risk.
What honest delivery looks like
Founders are sometimes told that their product is "fully tested" when what that means in practice is that unit tests pass. Those tests verify logic under controlled conditions. They do not verify real network delivery, lock-screen behaviour, OEM battery decisions, or whether environment variables are actually set correctly in production.
Honest delivery separates what was tested from what was not. It does not mean shipping nothing until everything is verified - that standard would prevent shipping most things. It means being explicit with your client or your team about the boundary.
Practically, this means: maintain a short list of pending verifications as a first-class project document. Update it when each item is resolved. Make it visible to the owner. That list - not the passing test count - is the most useful picture of actual readiness.
For a closer look at how the FCM and APNs integration was built in ManOps and what the tests covered, see push notifications: FCM and APNs gotchas. For the broader question of what a code or app health review actually checks, see ManOps - 5,000 tests: what they taught us.
The real-device call test script is listed in ManOps's project state document as the first item under "Not yet verified". If it were buried in a Slack thread or a to-do list that only the developer sees, the founder would have no way to know it was pending. Making it visible is a decision, not a side effect.
If you are a founder wondering whether your developer's "everything is tested" means what you think it means, request a technical review and we can show you exactly what was verified and what was not.
Common questions
What did ManOps testing not cover for push notifications and calls?
Zovintra had not verified real-phone FCM and APNs VoIP delivery, lock-screen ringing, answer-on-one-device cancelling the ring on another, or whether the APNs and FCM environment variables on Vercel deliver to real phones, as of 9 October 2026.
Is it acceptable to ship with documented untested areas?
For a small internal team with a limited blast radius, Zovintra considered it a reasonable trade-off, provided the limits are written down first. A documented list of pending verifications is a queue, not a risk. Undocumented unknowns are the dangerous kind.
How can I tell what my developer's 'everything is tested' really means?
Passing unit tests verify logic under controlled conditions, not real network delivery, lock-screen behaviour or production environment variables. Zovintra offers a technical review that can show exactly what was verified and what was not.
Sources
- ManOps project state document v1.4 (internal), 9 Oct 2026
- ManOps mobile platform notes (internal), 9 Oct 2026
