Privacy Review: Our Methodology

In our Privacy Reviews, we put apps and hardware to the test to see if any data is leaking – and why.

Our reviewers don’t interview the experts, they are the expert. We strongly believe looking under the hood is the only way to start an open dialogue about data privacy practices. We look at what an app is actually doing and document it.

As our reviewer Shoshana Wodinsky puts it: "If anybody ever asks for proof about what we found, I have screenshots, I have logs, I have it all."

Here’s a dive into our methodology, our reviewers, and our testing methods.

Type of Testing

Dynamic Network Traffic Interception and Penetration Testing. Our reviewers use Charles Proxy, a desktop network analysis tool that stands between the app on a smartphone and the destination servers. Every tap and background ping is captured in real-time, creating an immutable log with exact timestamps, recipient endpoints, and data payloads.

Our 10-Point Rating System

Every product we review is scored out of a maximum of 10 points across four categories. By our standards, a higher score means a more trustworthy, private product with good defaults and opt-out functionalities.

Default Settings

Great technology should do the heavy lifting for you, not create hurdles. We award the highest scores to products that treat transparency as a core design feature. This means that from the very first sign-up flow, the app makes it completely clear in plain English what it’s doing, how it’s doing it, and gives you immediate choice over your data.

Most users will take the path of least resistance, expecting an app to be safe to use in terms of data privacy. App and product designers know this, and they often use "malicious compliance" to satisfy legal frameworks while making actual privacy incredibly difficult to achieve.

We believe a truly user-first product doesn't require a law degree or advanced technical skills to navigate—it actively protects your privacy out of the box, allowing you to enjoy digital spaces safely and seamlessly from day one.

• 3 Points (Maximum Privacy): The app operates in the most private way possible by default without the user needing to change a single setting. For any internet-connected device, this means End-to-End Encryption (E2EE) is on by default. Your encryption keys stay on your local device, and the data is entirely unreadable by third parties—including the company providing the service.
• 2 Points (Hidden Controls): The app is capable of high-level privacy, but the settings are hard to locate or poorly flagged. Alternatively, the app may ask for tracking consent during onboarding but still sneakily trigger minor tracking background pings regardless of your choice.
• 1 Point (Non-Private & Exposed): The default configuration actively exposes personally identifiable information (PII) to the platform provider, and the necessary technical controls to turn off this data extraction are completely missing.
• 0 Points (Hostile Design): The product is structurally antithetical to user privacy. It spies on you by default, offers zero toggles to opt out, or utilizes hostile UX tricks to force data exposure. The product is unusable without giving up personal data.

Personal Data Collection

All apps need some data to operate, but data collection must be fit for purpose. Collecting basic analytics data or hardware identifiers (like a mobile ID) is structurally different from harvesting health data. Sharing a high volume of baseline app analytics is entirely distinct from leaking even a single, un-encryptable piece of highly sensitive personal health information.

• 3 Points (Non-Invasive): The app practices strict data minimization, gathering only what is necessary to deliver its core service.
• 2 Points (Vague or Intrusive Profiling): A point is knocked off if a product is intentionally vague about its data harvesting or collects data beyond what is reasonably necessary for the device to function.
• 1 Point (Discrepancies & Leaks): Major points are deducted if our dynamic network testing reveals a direct conflict between what the company promises in their privacy policy and what the code is actually doing on your device. Any hidden data processing or unlisted third-party transfers is an immediate red flag. This score also applies to apps that suffer from severe data leaks due to developer oversight, such as passing sensitive user data directly into an unencrypted network string to a third party without giving the user any chance to opt out.
• 0 Points (Data Selling / Invasive): If an app collects deeply invasive information that has no connection to functionality, provides absolutely no opt-out mechanisms for tracking, or sells user data, it receives an automatic 0. This includes apps that silently pipeline your behavioral data, device identifiers, and personal habits into third-party ad networks and data brokers without your explicit knowledge or consent.

Track Record

This metric evaluates corporate integrity based on two distinct lenses: Intent (is the company acting honestly?) and Competency (do they have the technical security skills to actually protect you?).

• 2 Points (Pristine History): The company has no record of major privacy scandals, no ongoing regulatory lawsuits, and its code handles data cleanly.
• 1 Point (Minor Oversights / Verifiable Effort): The company has faced security scrutiny or independent research reports in the past, but they have actively learned from their mistakes and patched the vulnerabilities.
• 0 Points (Deceptive or Incompetent): The company has a record of systemic, repetitive data breaches (showing poor competency), or there are reasonable and legitimate concerns raised about their privacy practices (showing poor intent).

Ability to Reduce Risks

For this metric we look at how easily you can protect your data while continuing to use the product.

• 2 Points (Trivial & Clear): Every tracking option and privacy toggle is located natively inside the app. The settings use straightforward, human-readable language instead of dense legal jargon. You can make the product safe to use in less than five minutes.
• 1 Point (Technical Effort / Covered Arses): The app features native privacy toggles, but hides them behind complex privacy policy jargon simply to cover the company legally. It requires advanced technical knowledge or external steps—like a user needing to set up an isolated, secondary Wi-Fi access point just to keep a smart appliance from snooping on the rest of their home network.
• 0 Points (Impossible / Functional Dead End): The app provides no internal controls and forces you outside of the product to protect your data (e.g., sending you to an external Google web page to opt out of analytics tracking). No points are given in our rating system when a device or app requires insecure cloud uploading to function at all (such as a cloud security camera that stops working entirely if you try to limit its cloud data streams), leaving you with no option to reduce your risk short of throwing the product in the trash.