CvInspect.Imaging 0.33.0

CvInspect.Imaging

Camera acquisition contract and tame frame sources for the CvInspect machine-vision toolkit, based on OpenCvSharp.

• ICam — the acquisition contract: open/close, single grab, continuous grab, connection/grabbing events, best-effort exposure control. Frames are GC-owned CamFrame buffers (byte[] + width/height/stride/format/timestamps) with no lifetime contract — frame.AsMat() gives a zero-copy Mat view for inspection, and the type implements the core ICvPixelSource contract, so CvDispCtrl (CvInspect.Wpf) displays it by reference without copying. A frame carries two clocks: TimestampUtc is when it arrived, and DeviceTimestamp is when the camera captured it (null if the device does not report one). The device clock has its own epoch, so compare frames to each other rather than reading it as wall time — the change in the gap between the two is time the frame spent waiting.
• VirtualCam — no-hardware source: a shifting-gradient test pattern, or name-ordered cyclic replay of an image folder (CamOpt.VirtualImageDir), re-enumerated on folder change. Ideal for development, demos and CI regression runs. Note: color→gray decoding follows OpenCV's decoder coefficients, so pixel values may differ slightly from images converted by other stacks (e.g. GDI luma) — build regression baselines through the same path.
• VideoCaptureCam — webcam (device index), video file (paced by FrameRate, loops at end) or stream URL (RTSP, …) through OpenCV VideoCapture (CamOpt.VideoSource).
• CamFactory — ComType string → implementation, with Register as the injection point for vendor SDK adapters (GigE etc.) so this package never references heavy SDKs.

Quick start

using CvInspect.Imaging;

var cam = CamFactory.Create(new CamOpt
{
    ComType = "Virtual",              // or "VideoCapture", or a registered vendor ComType
    VirtualImageDir = @"D:\samples",  // empty → test pattern
    FrameRate = 10,
});

cam.FrameAcquired += (_, frame) =>
{
    using var mat = frame.AsMat();   // zero-copy view into frame.Pixels
    // ... run CvInspect tools, update display ...
    // frame itself is GC-owned — keep it or hand it to another thread freely.
};

cam.Open();
cam.StartContinuous();

One frame, awaited

GrabOne() publishes through FrameAcquired and nothing else, so taking a single frame means subscribing, calling, waiting and unsubscribing. Every consumer wrote that by hand, which is a sign the library was missing something. GrabFrameAsync does it for you:

using CvInspect.Imaging;

var frame = await cam.GrabFrameAsync(TimeSpan.FromSeconds(2));   // GevCam throws TimeoutException on timeout — see below
if (frame is null) { /* no frame, and no reason to throw — see below */ }
else using (var mat = frame.AsMat()) { /* inspect */ }

GrabFrameAsync is an extension method, so it only appears on ICam when CvInspect.Imaging is imported — code that spells every type out in full will not see it without a using.

The frame also goes out on FrameAcquired — it is the same acquisition, and a display already subscribed should not miss this one frame. What you get back is that very instance, not merely some frame that arrived around the same time.

null versus an exception is the contract here. null means no frame for this call, and no reason to throw: the generic path's timeout expired, the implementation can never answer and has already logged why (DeadCam), or it received a frame it could not use and logged why (GevCam with an unsupported pixel format). null can come before the timeout, so do not read it as "timed out". Anything that should be able to answer but cannot — not open, closed, disposed, control lost, continuous acquisition running, another grab already waiting — throws, exactly as GrabOne does. Folding those into null would collapse the reason channel and leave the caller waiting for a frame that is never coming.

Timeouts are where backends differ. The generic path cannot tell why nothing came, so it returns null. GevCam knows where to look and throws a TimeoutException whose message points at what it logged at open (trigger mode on the camera-state line, and a separate warning when chunk mode is on) — the same body GrabOne runs. Code that must work on any backend treats null and TimeoutException alike as "no frame"; cancellation through the token comes out as OperationCanceledException, so it stays distinguishable from both.

When the call comes back with a frame, the grab is over — a GrabOne() or StartContinuous() issued right after it is not refused as "a single grab is already waiting". On the generic path the frame is published from inside GrabOne, so the call waits for GrabOne to return before it returns. It waits only up to the same timeout (a one-second grace with Timeout.InfiniteTimeSpan): if another FrameAcquired subscriber holds the frame longer than that, you get the frame at the timeout and the camera may still be finishing that grab. After a null or a cancellation the generic path cannot stop GrabOne either, so the next call can still be refused there. A camera that implements the grab itself keeps the promise on every outcome — it lowers its one-grab mark before its task completes (GevCam does). ReconnectingCam hands the call to its inner camera, so it keeps whatever that camera's path keeps: over VirtualCam or VideoCaptureCam, the generic path's.

If GrabOne throws, the call throws — even when a frame came in meanwhile. On the local cameras that means GrabOne refused at entry (continuous acquisition running, another grab waiting), so the frame the subscription picked up belonged to the live stream or to the other grab. Returning it would hide the refusal behind what looks like a normal answer. For the same reason a frame that arrives before this call's GrabOne has started is never taken — it cannot be this grab's answer.

The timeout argument wins over whatever the implementation has configured, because the call site knows more than the configuration did. Pass Timeout.InfiniteTimeSpan to defer to the implementation's own deadline — GevCam then uses its GrabTimeoutMs. On the generic path that deadline is GrabOne's own; if GrabOne comes back without a frame, a late one gets a one-second grace and then the call returns null.

Implement ICamGrabAsync when your camera knows something the generic path cannot, and the extension will call you instead. There are two such things:

• Which frame is this grab's. The generic path takes whatever arrives once its GrabOne has started; only the implementation, holding the device's own pairing evidence (frame id, ticket), can tell that from a frame that merely showed up at the same moment. GevCam does this with a start line latched from the device clock just before it starts acquisition, and falls back to the frame id on a camera that does not expose its clock.
• That no frame is coming at all. The generic path cannot distinguish "not yet" from "never", so it waits out the timeout rather than guessing — it will not cut short a backend that publishes just after GrabOne returns. (With Timeout.InfiniteTimeSpan there is no caller timeout left to wait out, so a late frame gets a one-second grace after GrabOne returns.) A camera that knows the answer immediately should say so: DeadCam returns null at once instead of stalling every grab for the full timeout.

GevCam, ReconnectingCam (which forwards to its inner camera) and DeadCam implement it.

Frame lifetime

CamFrame is a GC-owned byte[] holder — there is no lifetime contract: keep it, queue it, or post it to a UI dispatcher as-is. AsMat() wraps the pixel buffer without copying (disposing the view only unpins; the pixels stay valid), so running CvInspect tools costs no conversion. Sources materialize one buffer per frame; frames already have CamOpt.Flip / Rotation applied.

Crop(x, y, width, height) returns a new frame for a region — a copy with a tight stride that keeps the format and both timestamps (it is the same shot). The region must lie inside the frame; pass the rect CvImageOps.Crop reported, and move the result overlay with the same rect (ViOverlay.CropTo). On CvDispCtrl, set ClipOverlayToImage for such a view, or items outside the region are drawn in the fit margin.

Two more contract points implementers must honor: only complete frames are published (corrupt / partially received frames are dropped with a CvLog warning, never delivered), and Stride may exceed width × bytes-per-pixel on devices that pad rows — consumers must always walk rows by Stride (AsMat() handles this automatically).

USB webcams — a stable identity

OpenCV opens webcams by index, and the index is whatever order the OS enumerated the devices in this time. Re-plug a hub, reboot, add a capture card, and camera 0 and camera 1 swap without a warning. VideoCaptureCam therefore accepts an identity in CamOpt.SerialNumber:

new CamOpt { ComType = "VideoCapture", SerialNumber = "VID_046D&PID_082D" }

Accepted keys: VID_046D&PID_082D, 046D:082D, a full Windows instance id (USB\VID_046D&PID_082D\5&2C1F7A8&0&0001) or a Linux /dev/v4l/by-id name. When the key is not found, the exception lists every enumerated device so you can copy the right one into the settings; when two identical models match a VID/PID, it refuses to guess and lists their instance ids. UsbCamId.Enumerate() gives you the same list programmatically. UserSettings may carry backend=dshow|msmf|v4l2|any to pin the OpenCV backend.

Vendor adapters

Implement ICam in your application (or an adapter package) against the vendor SDK — materialize SDK buffers into CamFrame directly (or via CamFrame.FromMat) — and register it once at startup:

CamFactory.Register("MyGigE", opt => new MyGigECam(opt));

If the adapter will be wrapped in ReconnectingCam, two things must already be true when a call returns. Open must not return until IsConnected is true, because the wrapper checks it right after Open and treats false as a failed open. And a StartContinuous that started must have IsGrabbing true, because the wrapper checks that before it reports live. When the connection is lost, lower IsConnected (and IsGrabbing) before raising the events. Raise GrabbingChanged only on a transition, and catch your subscribers' exceptions on threads you own.

Subscribers: GrabbingChanged is a cue, not the state. It can arrive on a thread other than the one that called StartContinuous / StopContinuous — a control loss, a receive loop that ended on its own, the reconnect loop — and notifications raised on different threads can reach you out of order. If you marshal them to the UI asynchronously, the work they trigger can also land after other threads have already moved on (stopped the camera, started it again, or released whatever they held while acquiring). Re-read IsGrabbing where you decide; do not store the notification's value as your state.

Proprietary SDK assemblies stay out of this package by design.

Targets

netstandard2.1 and net8.0. Depends on CvInspect (managed OpenCvSharp4 flows transitively); the native OpenCV runtime is chosen by the consuming application — VideoCaptureCam additionally needs the videoio backend for your source type (the Windows runtime bundles it).

Versions

The four CvInspect packages move together — upgrade them as a set. This is a 0.x API: breaking changes happen, so pin versions. What changed in each release is on the Releases page, one entry per version; read the entries between the version you are on and the one you are moving to.

License

Apache-2.0. Not affiliated with OpenCV or OpenCvSharp.