Resources · Pilot guide
How a VisionAery pilot should be run
This document sets out the site requirements, the two-day proof of concept, the 30-day pilot and the pass criteria for each. It applies to every industry VisionAery serves: oil and gas, chemical, manufacturing, utilities, water, mining, data centers, energy storage, healthcare, commercial and retail, and municipal sites.
Stages and pass criteria
| Stage | What happens | Length | Pass criterion |
|---|---|---|---|
| 1. Proof of concept | Controlled releases on your site, by day and under IR at night, scored one event at a time | Two days on site | 80 percent or more of test events alarmed within 60 seconds of reaching the supported size |
| 2. 30-day pilot | The camera licensed on your site, running on real operations, alarms into your VMS or SCADA | 30 days from activation | Every confirmed release alarmed; false alarms at or under 1 per zone per day by day 30; nuisance alarms down by half after the first 72 hours and only under abnormal conditions by day 30; every alarm received where it was pointed |
The two stages run back to back: the proof of concept on the days the pilot camera is licensed, then thirty days of normal operation. One camera channel is covered by the free trial license; more cameras or sites are agreed in writing before day one. The decision is made at day 30.
Site requirements
These are checked at installation, before any test. A site that does not meet them is corrected first, not scored.
Camera and stream
- 1920 × 1080 or better, at 1 frame per second or better, RTSP with H.264.
- A fixed camera, or a PTZ parked on a preset. A PTZ can tour presets, and each preset carries its own zones.
- Image settings on automatic so the detection area stays in focus and correctly exposed through the day.
- At night, the detection area inside half of the camera's rated IR range. IR is not needed by day.
- A clean stream: no dropped packets, no compression artifacts.
Field of view
- Downward tilt of at least 5 degrees, 10 or more preferred. From 20 feet up, 10 degrees of tilt gives about 113 feet of usable range.
- Enough pixels on the ground. At 1080p, a scene 15 feet wide gives 128 pixels per foot, 30 feet gives 64, 40 feet gives 48. Releases under 15 gallons need 50 pixels per foot or more, 70 preferred. Larger releases need 30 or more.
Minimum puddle the camera is asked to see
| Scene width at the puddle | Pixels per foot at 1080p | Minimum puddle area |
|---|---|---|
| 15 ft | 128 | 4 sq ft |
| 20 ft | 96 | 6 sq ft |
| 30 ft | 64 | 10 sq ft |
| 40 ft | 48 | 15 sq ft |
| 60 ft | 32 | 25 sq ft |
| Over 60 ft | under 30 | Not supported |
A puddle smaller than the row for the scene width is not a test event. It may be detected; it is not scored.
Liquids
The requirement is contrast: the liquid has to look different from the surface it lands on, by eye, in the camera's image. Color is secondary.
- Supported: crude oil, produced water, fuels, hydraulic and lubricating oils, dark chemicals and coolants, and water darkened with dye, on light surfaces. Light liquids such as milk, light coolant or clear water on dark floors.
- Conditional: clear water on light surfaces, where the surface itself darkens when wet; lightly tinted liquids; any liquid close in tone to the surface.
- Not supported: liquids that cannot be told from the surface by eye in the camera image.
Surfaces
Three groups, by the evidence behind them.
- Field validated (operating sites): light-colored dirt and gravel that darken when wet, including Permian caliche; light-colored containment liner, stretched flat; uncoated concrete pads that darken when wet.
- Validated in controlled tests: sealed, polished and epoxy-coated concrete floors in plants, warehouses, pump rooms and data centers; store, terminal and corridor floors; asphalt and paved roads and lots; dark dirt or gravel that barely changes when wet; dark or wrinkled liner. On these surfaces the limiting factors are glare and reflections, so the lighting and camera-angle requirements matter more.
- Not supported: areas of heavy vegetation, mulch, wood, bare or painted metal, open grating, and any surface so reflective that the liquid cannot be told from the reflections.
The test area
- Largely unobstructed. Small objects such as pipes or cart wheels are acceptable if they do not hide the expected puddle.
- Able to hold a visible puddle for 15 seconds. Dirt and gravel need more liquid than a floor to get there.
- Free of stains from earlier releases, dark patches and other heavy discoloration.
- Evenly lit without harsh shadows, or unlit and covered by the camera's IR.
The full camera placement guide, with good and bad example images, is on camera requirements.
Stage 1: proof of concept, two days on site
VisionAery staff come to the site for two days, or the customer's crew runs the same tests with the same kit. Outdoors the kit is a water tank of 100 gallons or more, hoses, black pond dye and a pressure washer. Indoors it is a few gallons of the liquid the site handles, or water with a dye that contrasts with the floor, plus a continuous spray source for spray tests: a pressure washer or a hose with a mist nozzle. A hand spray bottle does not produce the sustained mist the model detects. Day one is the daytime test and the zone setup; the evening and day two repeat the tests under IR and on any spot where the camera was adjusted. Each intentional release is one test event, scored on its own.
- Mark the zones. Draw interest zones over the areas to be tested and exclusion zones over anything to ignore, exactly as they would be set for operation.
- Measure the scene. Record the scene width at each test spot and read the minimum puddle area from the table above. That is the size the release has to reach before the clock starts.
- Pour. Start with a sensitivity check of about a gallon, which may not reach the supported size and is not expected to alarm. Then the baseline test, three to five gallons, which reaches the supported size on most surfaces. Then a large event, six to fifteen gallons or more, to check speed and growth tracking. Outdoors, dark-dyed water is used because it behaves and reads like crude or produced water. Indoors, use the liquid the site handles where that is practical, otherwise water dyed to contrast with the floor.
- Spray. With the pressure washer, a short burst of 10 to 15 seconds and a sustained spray of 30 seconds or more, at more than one pressure and angle. Spray detection depends on the visual density of the mist and how long it lasts, not on the fact that a hose is running.
- Record each event. The moment the release reaches the supported size, the time of the first alarm, and the outcome. More than one alarm during a release still counts as one event.
- Repeat at night and on day two. The same tests under IR, since most sites are unattended after dark, and a second daytime pass on any spot where the camera was moved or re-zoomed.
How an event is scored
- Detected on time. At least one alarm within 60 seconds of the release reaching the supported size. The only outcome that counts toward the pass.
- Detected late. An alarm after 60 seconds. Tracked separately, because it usually points at camera placement, zoom or a threshold, and it tells us what to tune.
- Missed. No alarm. Recorded with a cause: low contrast, reflection, staining, obstruction, settings.
Pass criterion: 80 percent of test events detected on time. Puddle and spray events count alike.
The criterion is 80 percent rather than 100 because the proof of concept runs on a site the model has never seen, with no site-specific tuning or history, under the light and weather of the day. Eighty percent on a first visit shows that the camera, the surface and the model suit each other; operation improves from there as zones and thresholds are set for the site. Frame-level accuracy on the benchmark set and the field record are published separately; this criterion is about the site under test.
Stage 2: 30-day pilot
The pilot runs the camera on normal operations for thirty days from activation, with alarms delivered to the systems the site uses: the VMS through the camera's events, SCADA over MQTT, or a webhook. One camera channel is covered by the free trial license. A pilot across several cameras or sites is scoped in writing first: which sites, which cameras, which zones, which systems receive the alarms, and who confirms a release.
What is measured, and what counts as a pass
| Measure | How it is counted | Pass |
|---|---|---|
| Confirmed releases | A release verified on site inside an interest zone | Every one alarmed |
| False alarms | Alarms on something that is not liquid: a shadow, a stain, dark ground. Counted per zone per 24 hours | At or under 2 per zone per day in the first week, before site tuning; at or under 1 by day 30 |
| Nuisance alarms | Alarms on liquid that does not matter: a rain puddle, wash-down water. Counted per zone per 24 hours | Not scored in the first 48 to 72 hours, while the site has no baseline. Under half the first days' rate after that, and by day 30 only under abnormal conditions: heavy rain or snow, a new stain, a scenario the site had not seen |
| Integration | Alarms arriving in your system with the snapshot attached | Every alarm received where it was pointed, for the full period |
Nuisance alarms are expected on a new site until exclusion zones, alarm rules and, outdoors, the rain signal are set, and they fall once they are. Outdoors the rain sensor and the 24-hour post-rain forgiveness window are the largest single factor. Indoors the equivalents are wash-down schedules and exclusion zones over drains and mats. How the alarm logic keeps repeated alarms down is explained on AI leak detection.
These are the same measures as the field performance page, where ten tank batteries over ninety days produced one nuisance alarm per camera every 17 dry days and one false alarm every 41. A thirty-day pilot is judged the same way, so its result reads against that record.
What you get at day 30
A written record: every proof-of-concept event with its timestamps and outcome, the false and nuisance alarm counts per zone per day for the month, every alarm that reached your system, and a list of what was tuned and when. That record is the basis for the purchase decision.
Site footage in the model (optional)
No site-specific training is needed to run. With your permission, footage from the pilot can be added to the training set, and the model is updated on a thirty-day cycle. Sites that choose this see nuisance alarms fall further; most never need it.
Scorecard
Every test event is logged on the same twenty-field scorecard, so results from different sites can be compared. Fill it in during the test; it becomes the pilot record.
| # | Field | What goes in it |
|---|---|---|
| 1 | Site name or ID | Where the test ran |
| 2 | Camera ID or stream | The camera the analytic watched |
| 3 | Camera type | Fixed, or PTZ parked on a preset |
| 4 | Resolution and frame rate | For example 1920 × 1080 at 1 fps or more |
| 5 | Stream protocol | For example RTSP, H.264 |
| 6 | Camera height and tilt | Installation geometry, if known |
| 7 | Scene width at the test spot, feet | Measured, sets the minimum area |
| 8 | Minimum supported puddle area, square feet | From the table in section 3 |
| 9 | Zone | Which interest zone the event is in |
| 10 | Surface | Dirt, gravel, liner, concrete; supported or conditional |
| 11 | Liquid | Crude, produced water, dyed water, clear water, and the dye if any |
| 12 | Conditions | Day or night, shadows, rain, glare, wind |
| 13 | Event ID | One per pour or spray |
| 14 | Pour start | Timestamp |
| 15 | Supported size reached | Timestamp; the clock starts here |
| 16 | First alarm | Timestamp, if any |
| 17 | Outcome | Detected on time, detected late, missed |
| 18 | Detection time, seconds | First alarm minus supported size reached |
| 19 | Alarms during the event | A count; still one event |
| 20 | Notes and cause | For a late or missed event: contrast, reflection, stain, obstruction, settings, zoom |
Testing practices
- Use a liquid that contrasts with the surface.
- Check pixels per foot before the first test.
- Test on ground or floor with no old stains.
- Run several volumes and several spots, by day and under IR or the site's night lighting.
- Do not test with small amounts and expect an alarm. Visible area is what is detected, not volume.
- Do not test on reflective or unsupported surfaces.
- Do not ignore the field-of-view limits.
- Do not judge a thirty-day pilot on its first two days, before the site has a baseline.
Questions
How long does a VisionAery pilot take?
What does the pilot cost?
What is the pass criterion?
Why not 100 percent?
What is the difference between a false alarm and a nuisance alarm?
Does the system need to be trained on my site?
Can the customer run the proof of concept without VisionAery on site?
What if my cameras fail the site requirements?
What happens to the pilot record?
Next step
Two days on site, thirty days of operation, the record and the decision at day thirty.