MediaStreamTrack.
The API defined in this document captures images from a photographic device referenced through a valid [[!GETUSERMEDIA]] MediaStreamTrack. The produced image can be in the form of a [[!FILEAPI]] Blob (see takePhoto() method) or as a [[!HTML51]] ImageBitmap (see grabFrame()). Picture-specific settings can be optionally provided as arguments that can be applied to the device for the capture.
The User Agent must support Promises in order to implement the Image Capture API. Any Promise object is assumed to have resolver object, with resolve() and reject() methods associated with it.
[Constructor(MediaStreamTrack track)]
interface ImageCapture {
readonly attribute MediaStreamTrack videoStreamTrack;
Promise<Blob> takePhoto ();
Promise<PhotoCapabilities> getPhotoCapabilities ();
Promise<void> setOptions (PhotoSettings? photoSettings);
Promise<ImageBitmap> grabFrame ();
};
takePhoto() method returns a captured image encoded in the form of a Blob, whereas grabFrame() returns a snapshot of the `MediaStreamTrack` video feed in the form of a non encoded ImageBitmap.
ImageCapture| Parameter | Type | Nullable | Optional | Description |
|---|---|---|---|---|
| track | MediaStreamTrack |
✘ | ✘ | The `MediaStreamTrack` to be used as source of data. This will be the value of the videoStreamTrack attribute. The `MediaStreamTrack` passed to the constructor MUST have its `kind` attribute set to "video" otherwise a DOMException of type NotSupportedError will be thrown. |
videoStreamTrack of type MediaStreamTrack, readonlytakePhototakePhoto() produces the result of a single photographic exposure using the video capture device sourcing the videoStreamTrack, applying any PhotoSettings previously configured, and returning an encoded image in the form of a Blob if successful. When this method is invoked:
readyState of the MediaStreamTrack provided in the constructor is not `live`, return a promise rejected with a new DOMException ([[!WebIDL]]) whose name is "InvalidStateError". Otherwise:MediaStreamTrack into a Blob containing a single still image. The method of doing this will depend on the underlying device. Devices may temporarily stop streaming data, reconfigure themselves with the appropriate photo settings, take the photo, and then resume streaming. In this case, the stopping and restarting of streaming SHOULD cause mute and unmute events to fire on the Track in question. takePhoto() method for any reason (for example, upon invocation of multiple takePhoto() method calls in rapid succession), then the UA MUST return a promise rejected with a new DOMException ([[!WebIDL]]) whose name is "UnknownError".getPhotoCapabilitiesgetPhotoCapabilities() is used to retrieve the ranges of available configuration options and their current setting values, if any. When this method is invoked:
readyState of the MediaStreamTrack provided in the constructor is not `live`, return a promise rejected with a new DOMException ([[!WebIDL]]) whose name is "InvalidStateError". MediaStreamTrack into a PhotoCapabilities object containing the available capabilities of the device, including ranges where appropriate. The resolved PhotoCapabilities will also include the current conditions in which the capabilities of the device are found. The method of doing this will depend on the underlying device. getPhotoCapabilities() method for any reason (for example, the MediaStreamTrack being ended asynchronously), then the UA MUST return a promise rejected with a new DOMException ([[!WebIDL]]) whose name is "OperationError".PhotoCapabilities object.setOptionssetOptions() is used to configure a number of settings affecting the image capture and/or the current video feed in videoStreamTrack. When this method is invoked:
readyState of the MediaStreamTrack provided in the constructor is not `live`, return a promise rejected with a new DOMException ([[!WebIDL]]) whose name is "InvalidStateError". PhotoSettings object is passed as argument, return a promise rejected with a new DOMException ([[!WebIDL]]) whose name is "SyntaxError"settings parameter.DOMException ([[!WebIDL]]) whose name is "OperationError". videoStreamTrack.
| Parameter | Type | Nullable | Optional | Description |
|---|---|---|---|---|
| settings | PhotoSettings |
✔ | ✘ |
The PhotoSettings dictionary to be applied.
|
grabFramegrabFrame() takes a snapshot of the live video being held in the videoStreamTrack, returning an ImageBitmap if successful. When this method is invoked:
readyState of the MediaStreamTrack provided in the constructor is not `live`, return a promise rejected with a new DOMException ([[!WebIDL]]) whose name is "InvalidStateError". Otherwise:MediaStreamTrack into an ImageBitmap object (as defined in [[!HTML51]]). The width and height of the ImageBitmap object are derived from the constraints of the MediaStreamTrack. ImageBitmap object. (Note: grabFrame() returns data only once upon being invoked).grabFrame() method for any reason (for example, upon invocation of multiple grabFrame()/takePhoto() method calls in rapid succession), then the UA MUST return a promise rejected with a new DOMException ([[!WebIDL]]) whose name is "UnknownError".PhotoCapabilities
interface PhotoCapabilities {
readonly attribute MeteringMode whiteBalanceMode;
readonly attribute MediaSettingsRange colorTemperature;
readonly attribute MeteringMode exposureMode;
readonly attribute MediaSettingsRange exposureCompensation;
readonly attribute MediaSettingsRange iso;
readonly attribute boolean redEyeReduction;
readonly attribute MeteringMode focusMode;
readonly attribute MediaSettingsRange brightness;
readonly attribute MediaSettingsRange contrast;
readonly attribute MediaSettingsRange saturation;
readonly attribute MediaSettingsRange sharpness;
readonly attribute MediaSettingsRange imageHeight;
readonly attribute MediaSettingsRange imageWidth;
readonly attribute MediaSettingsRange zoom;
readonly attribute FillLightMode fillLightMode;
};
whiteBalanceMode of type MeteringModecolorTemperature of type MediaSettingsRangeexposureMode of type MeteringModeexposureCompensation of type MediaSettingsRangeiso of type MediaSettingsRangeredEyeReduction of type booleanfocusMode of type MeteringModebrightness of type MediaSettingsRangecontrast of type MediaSettingsRangesaturation of type MediaSettingsRangesharpness of type MediaSettingsRangeimageHeight of type MediaSettingsRangeimageWidth of type MediaSettingsRangezoom of type MediaSettingsRangefillLightMode of type FillLightModeFillLightMode.imageWidth and imageHeight ranges to prevent increasing the fingerprinting surface and to allow the UA to make a best-effort decision with regards to actual hardware configuration.
The PhotoCapabilities interface provides the photo-specific settings options and current settings values. The following definitions are assumed for individual settings and are provided for information purposes:
| Mode | Kelvin range |
|---|---|
| incandescent | 2500-3500 |
| fluorescent | 4000-5000 |
| warm-fluorescent | 5000-5500 |
| daylight | 5500-6500 |
| cloudy-daylight | 6500-8000 |
| twilight | 8000-9000 |
| shade | 9000-10000 |
PhotoSettingsThe PhotoSettings object is optionally passed into the setOptions() method in order to modify capture device settings specific to still imagery. Each of the attributes in this object is optional.
dictionary PhotoSettings {
MeteringMode whiteBalanceMode;
unsigned long colorTemperature;
MeteringMode exposureMode;
unsigned long exposureCompensation;
unsigned long iso;
boolean redEyeReduction;
MeteringMode focusMode;
sequence<Point2D> pointsOfInterest;
unsigned long brightness;
unsigned long contrast;
unsigned long saturation;
unsigned long sharpness;
unsigned long zoom;
unsigned long imageHeight;
unsigned long imageWidth;
FillLightMode fillLightMode;
};
whiteBalanceMode of type MeteringModecolorTemperature of type unsigned longmanual.exposureMode of type MeteringModeexposureCompensation of type unsigned long, multiplied by 100 (to avoid using floating point). iso of type unsigned longredEyeReduction of type booleanfocusMode of type MeteringModepointsOfInterest of type sequence<Point2D>sequence of Point2Ds to be used as metering area centers for other settings, e.g. Focus, Exposure and Auto White Balance.brightness of type unsigned longcontrast of type unsigned longsaturation of type unsigned longsharpness of type unsigned longzoom of type unsigned longimageHeight of type unsigned longimageWidth of type unsigned longfillLightMode of type FillLightModeFillLightMode.MediaSettingsRange
interface MediaSettingsRange {
readonly attribute long max;
readonly attribute long min;
readonly attribute long current;
readonly attribute long step;
};
max of type long, readonlymin of type long, readonlycurrent of type long, readonlystep of type long, readonlyFillLightMode
enum FillLightMode {
"unavailable",
"auto",
"off",
"flash",
"torch"
};
unavailableautoflash to guarantee firing of the flash for the takePhoto() or getFrame() methods.offflashtakePhoto() or getFrame() methods. torchMediaStreamTrack is activeMeteringModeNote that MeteringMode is used for both status enumeration and for setting options for capture(s).
enum MeteringMode {
"none",
"manual",
"single-shot",
"continuous"
};
nonemanualsingle-shotcontinuousA Point2D represents a location in a normalized square space with values in `[0.0, 1.0]`. The origin of coordinates `(0.0, 0.0)` represents the upper leftmost corner, with the `y` coordinate pointing downwards.
Point2D
dictionary Point2D {
float x = 0.0;
float y = 0.0;
};
x of type floaty of type floattakePhoto()
<html>
<body>
<p><video/></p>
<p><img/></p>
<input type="range">
<script>
var imageCapture;
navigator.mediaDevices.getUserMedia({video: true})
.then(gotMedia)
.catch(e => { console.error('getUserMedia() failed: ', e); });
function gotMedia(mediastream) {
const track = mediastream.getVideoTracks()[0];
imageCapture = new ImageCapture(track);
imageCapture.getPhotoCapabilities().then(photoCapabilities => {
// Check whether zoom is supported or not.
if (!photoCapabilities.zoom.min && !photoCapabilities.zoom.max) {
return;
}
// Map zoom to a slider element.
const input = document.querySelector('input[type="range"]');
input.min = photoCapabilities.zoom.min;
input.max = photoCapabilities.zoom.max;
input.step = photoCapabilities.zoom.step;
input.value = photoCapabilities.zoom.current;
input.oninput = function(event) {
imageCapture.setOptions({zoom: event.target.value});
}
})
.catch(e => {
console.error("Uh, oh, getPhotoCapabilities() failed: ", e);
});
}
function takePhoto() {
imageCapturer.takePhoto()
.then((blob) => {
console.log("Photo taken: " + blob.type + ", " + blob.size + "B")
const imageTag = document.querySelector('input[type="img"]');
imageTag.src = URL.createObjectURL(blob);
})
.catch((err) => {
console.error("takePhoto() failed: ", e);
});
}
</script>
</body>
</html>
##### Repeated grabbing of a frame
<html>
<body>
<p><canvas id="canvas"></canvas></p>
<button onclick="stopFunction()">Stop frame grab</button>
<script>
'use strict';
var interval;
var canvas = document.getElementById('canvas');
var videoTrack;
navigator.mediaDevices.getUserMedia({video: true})
.then(gotMedia)
.catch(e => { console.error('getUserMedia() failed: ', e); });
function gotMedia(mediastream) {
videoTrack = mediastream.getVideoTracks()[0];
var imageCapture = new ImageCapture(videoTrack);
interval = setInterval(function () {
imageCapture.grabFrame()
.then(processFrame)
.catch(e => console.error('grabFrame() failed: ' + e));
}, 1000);
}
function processFrame(imgData) {
canvas.width = imgData.width;
canvas.height = imgData.height;
canvas.getContext('2d').drawImage(imgData, 0, 0);
}
function stopFunction(e) {
clearInterval(interval);
videoTrack.stop();
}
</script>
</body>
</html>
##### Grabbing a Frame and Post-Processing
<html>
<body>
<p><canvas id="canvas"></canvas></p>
<script>
var canvas = document.getElementById('canvas');
var videoTrack;
navigator.mediaDevices.getUserMedia({video: true})
.then(gotMedia)
.catch(e => { console.error('getUserMedia() failed: ', e); });
function gotMedia(mediastream) {
videoTrack = mediastream.getVideoTracks()[0];
var imageCapture = new ImageCapture(videoTrack);
imageCapture.grabFrame()
.then(processFrame(imgData))
.catch(e => console.error('grabFrame() failed: ' + e));
}
function processFrame(imageBitmap) {
videoTrack.stop();
// |imageBitmap| pixels are not directly accessible: we need to paint
// the grabbed frame onto a <canvas>, then getImageData() from it.
var ctx = canvas.getContext('2d');
canvas.width = imageBitmap.width;
canvas.height = imageBitmap.height;
ctx.drawImage(imageBitmap, 0, 0);
// Read back the pixels from the <canvas>, and invert the colors.
var imageData = ctx.getImageData(0,0,canvas.width, canvas.height);
var data = imageData.data;
for (var i = 0; i < data.length; i += 4) {
data[i] = 255 - data[i]; // red
data[i + 1] = 255 - data[i + 1]; // green
data[i + 2] = 255 - data[i + 2]; // blue
}
// Finally, draw the inverted image to the <canvas>
ctx.putImageData(imageData, 0, 0);
}
</script>
</body>
</html>