# Spotting fake vaccines with black tape, a phone and two models
Here's the whole test: put a 10 µL drop of a vaccine on black electrician's tape, wait two hours, and take a photo with your phone. The tape costs less than a penny per test. No lab, no device, no training.
The pipeline.
It works because liquids don't dry the same way. What's dissolved in them, salts, sugars, proteins, leaves a pattern behind. A real vaccine and a fake one made of saline or glucose leave visibly different spots.
This was a project with the [Vaccine Identity Evaluation](https://www.cghr.ox.ac.uk/research/medicine-quality-research-group/mqrg-projects/vaccine-identity-evaluation) team at Oxford. The [preprint](https://ssrn.com/abstract=5028576) is written for the medical world. This is the developer version.
### Why anyone needs this
Fake vaccines are not hypothetical. During COVID there were reports from dozens of countries of "vaccines" that were really water, saline, glucose or even antibiotics. Criminals also refill genuine vials with genuine labels, so checking the packaging isn't enough. You have to check the liquid.
The tools that can do that, like Raman spectroscopy or mass spectrometry, are expensive and live in labs. The places that need screening most, at the far end of the supply chain, don't have labs.
The idea itself came from an accident: while analysing vaccines on mass spectrometry plates, the team noticed that each vaccine dried with its own pattern and texture.
### What the spots look like
We tested 14 liquids: 6 vaccines, 2 insulins, and 6 stand-ins for fakes that have actually been found in the wild (tap water, saline, glucose, a hyaluronic acid serum and two antibiotics).
Top: fakes (1–2 pure water, 3 tap water, 4 saline, 5 glucose, 6 hyaluronic acid serum, 7 amikacin, 8 gentamicin). Bottom: vaccines (9–14) and insulin (15–16).
Some are easy. Pure water leaves nothing at all, which is a result in itself. Saline leaves salt crystals around the border. Tap water leaves the classic "coffee ring": as the drop evaporates from the edge, liquid flows outward and carries the minerals with it. Glucose dries into a small, hard, shiny disc.
Two people looking at the spots could tell every liquid apart, but only if they had also seen the drop *before* it dried (some were cloudy or white). From the dried spot alone, two pairs were impossible: the insulins Novorapid and Insulatard, and the flu vaccines Fluad and Flucelvax. Those pairs are where AI had to earn its place.
### Two models, again
Like in [CiliAI](/post/ciliai-small-object-detection/), the pipeline is two models instead of one:
1. **Mask R-CNN (Detectron2)** finds the spot. It's trained on a single class, `spot`, and doesn't care what the spot is.
2. The spot is cropped out using that mask, and **EfficientNet-B5** classifies the crop into one of the 14 liquids.
The split pays off twice. A one-class segmentation model is an easy job, so it generalises well: a spot is a spot. And the classifier only ever sees the spot itself, not the tape around it, the table, the edge of the photo or the phone's colour processing. It can't take shortcuts by learning the background.
### Small data
We had 380 photos in total, about 25–30 per liquid. That's tiny. Instead of more photos of the same thing, we went for *variation*: two phone models, two photographers, three lighting conditions, several resolutions and aspect ratios. Then augmentation on top: rotation, contrast and lighting changes.
The photos were annotated with polygons, and 84 of them (6 per liquid) were set aside as a test set the models never saw during training.
If you're working with small datasets, this is the part I'd copy. The question isn't "how many images do I have" but "how many of the ways the real world will differ have I shown the model".
### Results
- **Segmentation:** mean IoU of 0.936 on the test set. The worst class was glucose (0.900): shiny spots with reflections have fuzzier edges.
- **Classification:** 84 out of 84 correct, 100% accuracy, including the insulin and flu vaccine pairs that humans couldn't separate from the dried spot alone.
The more interesting number is confidence. Three test images were classified correctly but with confidence below 0.8: Novorapid vs Insulatard, Nimenrix vs Insulatard, and glucose vs gentamicin. One glucose image was correct with a confidence of just 0.45. "Correct" and "sure" are not the same thing, and an app built on this should show the confidence, not just the label.
### Why I don't fully trust our 100%
A perfect score on 84 images is good news, but it mostly tells you what you *didn't* test.
- **One room.** Every spot dried in the same lab, at about 20 °C and 50% humidity, under the same fluorescent ceiling lights. Spots may dry differently at 35 °C and 90% humidity, which is exactly where screening is needed.
- **Same vials.** Training and test photos came from the same vials and syringes, the same tape brand and the same two phones. The test set proves the pipeline works *here*; it doesn't prove it works *everywhere*.
- **Reflections.** The shiny spots reflected the ceiling lights as bright white lines, and all of the shiny ones happen to be fakes. A model could learn "reflection = fake" instead of anything about the spot. It didn't seem to matter here, but under different lighting it could.
- **No substandard samples.** We tested fakes, not genuine vaccines with too little active ingredient. Those could look much closer to the real thing.
None of this makes the result wrong. It makes it a proof of concept, which is what it is.
### What's next
The obvious next step is a phone app that works offline: photograph the spot, get a result with a confidence score. A few ideas for making it better:
- **Size.** The model never knew the spot's diameter, and some liquids differ there. The tape is 25 mm wide, which makes it a free ruler; newer phones with LiDAR can measure distance directly.
- **Two photos.** One before drying adds information the models never saw: cloudiness and colour.
- **Local libraries.** Each country, or each regulator, could build its own reference images under its own climate and lighting, ideally as a shared international dataset.
What I like most about this project is the cost: a strip of tape and a phone most people already have. The AI is the cheap part.
- Preprint: [Dried spot analysis on SSRN](https://ssrn.com/abstract=5028576)
