Introduction

The MediaStream interface, as defined in the [[!GETUSERMEDIA]] specification, typically represents a stream of data of audio and/or video. This object may be extended to also represent a stream that either comes from or is sent to a remote node (and not just the local camera, for instance). The extensions required to enable this capability on the MediaStream object will be described in this document.

A MediaStream as defined in [[!GETUSERMEDIA]] may contain zero or more MediaStreamTrack objects. A MediaStreamTrack sent to another peer MUST appear as one and only one MediaStreamTrack to the recipient.

Channels are the smallest unit considered in the MediaStream specification. Channels are intended to be encoded together for transmission as, for instance, an RTP payload type. All of the channels that a codec needs to encode jointly MUST be in the same MediaStreamTrack and the codecs SHOULD be able to encode, or discard, all the channels in the track.

The concepts of an input and output to a given MediaStream apply in the case of MediaStream objects transmitted over the network as well. A MediaStream created by a PeerConnection object (later described in this document) will take as input the data received from a remote peer. Similarly, a MediaStream from a local source, for instance a camera via [[!GETUSERMEDIA]] will have an output that represents what is transmitted to a remote peer if the object is used with a PeerConnection object.

The concept of duplicating MediaStream objects as described in [[!GETUSERMEDIA]] is also applicable here. This feature can be used, for instance, in a video-conferencing scenario to display the local video from the user’s camera and microphone in a local monitor, while only transmitting the audio to the remote peer (e.g. in response to the user using a "video mute" feature). Combining tracks from different MediaStream objects into a new MediaStream is useful in certain cases.

Interface definitions

In this section, we only specify aspects of the the following objects that are relevant when used along with a PeerConnection. Please refer to the original definitions of the objects in the [[!GETUSERMEDIA]] document for general information on using MediaStream and MediaStreamTrack both in and outside the context of PeerConnection.

AudioMediaStreamTrack

The AudioMediaStreamTrack is a specialization of of a normal MediaStreamTrack that only carries audio and is extended to have the capability to send and/or receive DTMF codes.

readonly attribute boolean canInsertDTMF

The canInsertDTMF attribute MUST indicate if the AudioMediaStreamTrack is capable of sending DTMF.

void insertDTMF(in DOMString tones, optional long duration)

When a AudioMediaStreamTrack object’s insertDTMF() method is invoked, the user agent MUST queue a task that that sends the DTMF tones.

The tone parameters is treated as a series of characters. The characters 0 to 9, A to D, #, and * generated the associated DTMF tones. The characters a to d are equivalent to A to D. The character , indicates a an delay of 2 seconds before processing the next character in the tones parameter. Unrecognized characters are ignored.

The duration parameters indicates the duration in ms to play the each DTMF passed in the tones parameters. The duration can not be more than 6000 or less than 70. The default duration is 100 ms for each tone. The gap between tones MUST be at least 50 ms but should be as short as possible.

If insertDTMF is called on the same object while an existing task for this object is generate DTMF is still running, the previous task is canceled. Calling insertDTMF with an empty tones parameter can be used to cancel any tones currently being send.

Editor Note: We need to add a callback that is set on the object that is called after the tones are sent. This is needed to allow the application to know when it can send new tones without canceling the tones that are currently being sent.

Editor Note: It seems we would want a callback or event for incoming tones. The proposal sent to the list had them played as audio to the speaker but I don’t see how that is useful.

PeerConnection

>>>>>> master class='idl'>

void processSignalingMessage (DOMString message)

When a message is relayed from the remote peer over the signaling channel is received by the Web application, pass it to the user agent by calling the processSignalingMessage() method.

The order of messages is important. Passing messages to the user agent in a different order than they were generated by the remote peer’s user agent can prevent a successful connection from being established or degrade the connection’s quality if one is established.

When the processSignalingMessage() method is invoked, the user agent MUST run the following steps:

1. Let message be the method’s argument.

2. Let connection be the PeerConnection object on which the method was invoked.

3. If connection’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

4. If the first four characters of message are not "SDP" followed by a U+000A LINE FEED (LF) character, then abort these steps. (This indicates an error in the signaling channel implementation. User agents MAY report such errors to their developer consoles to aid debugging.)

   Future extensions to the PeerConnection interface might use other prefix values to implement additional features.

5. Let sdp be the string consisting of all but the first four characters of message.

6. Pass the sdp to the PeerConnection SDP Agent as a subsequent offer or answer, to be interpreted as appropriate given the current state of the SDP Agent. [[!ICE]]

When a PeerConnection ICE Agent forms a connection to the the far side and enters the state ICE_CONNECTED, the user agent MUST queue a task that sets the PeerConnection object’s PeerConnection readiness state to ACTIVE (2) and then fires a simple event named open at the PeerConnection object.

const unsigned short NEW = 0

The object was just created and its ICE and SDP Agent have not yet been started.

const unsigned short NEGOTIATING = 1

The peerConenction object is attempting to get to the point wehre media can flow.

const unsigned short ACTIVE = 2

A connection has been formed and if any media streams were successfully negotiated, any relevant media can be streaming.

const unsigned short CLOSING = 4

The object is starting to shut down after the close() method has been invoked.

const unsigned short CLOSED = 3

The close() method has been invoked.

readonly attribute unsigned short readyState

The readyState attribute MUST return the PeerConnection object’s PeerConnection readiness state, represented by a number from the following list:

PeerConnection . NEW (0)

The object was just created, and no networking has yet occurred.

PeerConnection . NEGOTIATING (1)

The user agent is attempting to establish an connection with the ICE Agent and to negotiate codecs with the SDP Agent.

PeerConnection . ACTIVE (2)

The ICE Agent has found a connection the SDP Agent has performed a round of codec negotiation. It is possible for whatever media was negotiated to flow.

PeerConnection . CLOSING (4)

The PeerConnection object is terminating all media and is in the process of closing the Ice Agent and SDP Agent.

PeerConnection . CLOSED (3)

The connection is closed.

const unsigned short ICE_GATHERING = 0x100

The ICE Agent is gather addresses that can be used.

const unsigned short ICE_WAITING = 0x200

THE ICE Agent has complete gathering addresses and is waiting for candidates to start checking.

const unsigned short ICE_CHECKING = 0x300

The ICE Agent is checking candidates but has not yet found a connection that works.

const unsigned short ICE_CONNECTED = 0x400

The ICE Agent has found at least one candidate that works but is still checking.

const unsigned short ICE_COMPLETED = 0x500

The ICE Agent has finished checking all candidates and a connection has been formed.

const unsigned short ICE_FAILED = 0x600

The ICE Agent has finished checking all candidates and no connection was worked.

const unsigned short ICE_CLOSED = 0x700

The ICE Agent is terminating and will no longer repined to STUN connectivity checks.

readonly attribute unsigned short iceState

The iceState attribute MUST return the state of the PeerConnection ICE Agent ICE state, represented by a number from the following list:

PeerConnection . NEW (0)

The object was just created, and no networking has yet occurred.

PeerConnection . ICE_GATHERING (0x100)

The ICE Agent is attempting to establish a gather addresses.

PeerConnection . ICE_WAITING (0x200)

The ICE Agent is waiting for candidates from the other side before it can start checking.

PeerConnection . ICE_CHECKING (0x300)

The ICE Agent is checking candidates but has not yet found a connection.

PeerConnection . ICE_CONNECTED (0x400)

The ICE Agent has found a connection but is still checking other candidates to see if there is a better connection.

PeerConnection . ICE_COMPLETED (0x500)

The ICE Agent has finished checking and found a connection.

PeerConnection . ICE_FAILED (0x600)

The ICE Agent is finished checking all candidates and failed to find a connection.

PeerConnection . ICE_CLOSED (0x700)

The ICE Agent has shut down and is no longer responding to STUN requests.

const unsigned short SDP_IDLE = 0x1000

A valid offer anser pair has been exchanged and the SDP Agent is waiting for the next SDP transaction.

const unsigned short SDP_WAITING = 0x2000

The SDP Agent has sent an SDP offer and is waiting for a response.

const unsigned short SDP_GLARE = 0x3000

Both side sent SDP offers at the same time and the SDP Agent is waiting to be able to retransmit the SDP offer.

readonly attribute unsigned short sdpState

The sdpState attribute MUST return the state of the PeerConnection SDP Agent , represented by a number from the following list:

PeerConnection . NEW (0)

The object was just created, and no networking has yet occurred.

PeerConnection . SDP_IDLE (0x1000)

At least one SDP offer or answer has been exchange and the SDP Agent is ready to send an SDP offer or receive an SDP answer.

PeerConnection . SDP_WAITING (0x2000)

The SDP Agent has sent and offer and is waiting for a answer.

PeerConnection . SDP_GLARE (0x3000)

The SDP Agent received an offer while waiting for an answer and now much wait a random amount of time before retrying to send the offer.

DataChannel createDataChannel([TreatNullAs=EmptyString] DOMString? label, optional DataChannelInit? dataChannelDict)

Creates a new DataChannel object with the given label. The DataChannelInit dictionary can be used to configure properties of underlying channel such as data reliability. A corresponding DataChannel object is dispatched at the other peer if the channel setup was successful.

When the createDataChannel() method is invoked, the user agent MUST run the following steps.

1. If the PeerConnection object’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

2. Let channel be a newly created DataChannel object.

3. Initialize channel’s label attribute to the value of the first argument.

4. Initialize channel’s reliable attribute to true.

5. If the second argument is present and it contains a reliable dictionary member, then set channel’s reliable attribute to the dictionary member value.

6. Return channel and continue these steps in the background.

7. Create channel’s associated underlying data transport.

attribute Function? ondatachannel

This event handler, of type datachannel, MUST be supported by all objects implementing the PeerConnection interface.

void addStream (MediaStream stream, MediaStreamHints hints)

Attempts to starting sending the given stream to the remote peer. The format for the MediaStreamHints objects is currently undefined by the specification.

When the other peer starts sending a stream in this manner, an addstream event is fired at the PeerConnection object.

When the addStream() method is invoked, the user agent MUST run the following steps:

1. Let stream be the method’s first argument.

2. Let hints be the method’s second argument.

3. If the PeerConnection object’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

4. If stream is already in the PeerConnection object’s localStreams object, then abort these steps.

5. Add stream to the end of the PeerConnection object’s localStreams object.

6. Return from the method.

7. Parse the hints provided by the application and apply them to the MediaStream, if possible.

8. Have the PeerConnection add a media stream for stream the next time the user agent provides a stable state. Any other pending stream additions and removals MUST be processed at the same time.

void removeStream (MediaStream stream)

Stops sending the given stream to the remote peer.

When the other peer stops sending a stream in this manner, a removestream event is fired at the PeerConnection object.

When the removeStream() method is invoked, the user agent MUST run the following steps:

1. Let stream be the method’s argument.

2. If the PeerConnection object’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

3. If stream is not in the PeerConnection object’s localStreams object, then abort these steps.

4. Remove stream from the PeerConnection object’s localStreams object.

5. Return from the method.

6. Have the PeerConnectionremove the media stream for stream the next time the user agent provides a stable state. Any other pending stream additions and removals MUST be processed at the same time.

readonly attribute MediaStream[] localStreams

Returns a live array containing the streams that the user agent is currently attempting to transmit to the remote peer (those that were added with addStream()).

Specifically, it MUST return the read-only MediaStream array that the attribute was set to when the PeerConnection’s constructor ran.

readonly attribute MediaStream[] remoteStreams

Returns a live array containing the streams that the user agent is currently receiving from the remote peer.

Specifically, it MUST return the read-only MediaStream array that the attribute was set to when the PeerConnection’s constructor ran.

This array is updated when addstream and removestream events are fired.

void close ()

When the close() method is invoked, the user agent MUST run the following steps:

1. If the PeerConnection object’s PeerConnection readiness state is CLOSED (3), throw an INVALID_STATE_ERR exception.

2. Destroy the PeerConnection ICE Agent, abruptly ending any active ICE processing and any active streaming, and releasing any relevant resources (e.g. TURN permissions).

3. Set the object’s PeerConnection readiness state to CLOSED (3).

The localStreams and remoteStreams objects remain in the state they were in when the object was closed.

attribute Function? onconnecting

This event handler, of event handler event type connecting, MUST be supported by all objects implementing the PeerConnection interface.

attribute Function? onopen

This event handler, of event handler event type open, MUST be supported by all objects implementing the PeerConnection interface.

attribute Function? onstatechange

This event handler, of event handler event type open, MUST be supported by all objects implementing the PeerConnection interface. It is called any time the readyState, iceState, or sdpState changes.

attribute Function? onaddstream

This event handler, of event handler event type addstream, MUST be supported by all objects implementing the PeerConnection interface.

attribute Function? onremovestream

This event handler, of event handler event type removestream, MUST be supported by all objects implementing the PeerConnection interface.

Constraints

The following new constraints are defined that can be used with a PeerConnection object:

ReceiveVideo

This is a enum type constraint that can take the values "true" and "false". The default is a non mandatory "true" if the PeerConnection a video stream has been added no "false" otherwise.

In some cases, a PeerConenction may wish to receive video but it is not going to send any video. The PeerConenction needs to know if it should signal to the remote side if it wishes to receive video or not. This constraints allows an application to indicate it's preferences for receiving video when creating an offer.

VoiceActivityDetection

This is a enum type constraint that can take the values "true" and "false". The default is a non mandatory "true".

Many codecs and system are capable of detecting "silence" and changing there behavior in this case by doing things such as not transmitting any media. In many cases, such as when dealing with sounds other than spoken voice or emergency calling, it is desirable to be able to turn off this behavior. This constraints allows the application to provide information about if it wishes this type of processing enable or disabled.

MediaTransportRelayOnly

This is a enum type constraint that can take the values "true" and "false". The default is a non mandatory "false".

This constraints indicat4es if the ICE engine is restricted to only using media relay candidates such as candidates passing through a TURN server. This can be used to reduce leakage of IP addresses in certain use cases.

TODO items - need to register with IANA. Need to have error on constructor if not possible to resolve them.

SignalingCallback

void handleEvent (DOMString message, PeerConnection source)

Def TBD

Examples

// the first argument describes the STUN/TURN server configuration
var local = new PeerConnection('TURNS example.net', sendSignalingChannel);
local.signalingChannel(...); // if we have a message from the other side, pass it along here

// (aLocalStream is some LocalMediaStream object)
local.addStream(aLocalStream); // start sending video

function sendSignalingChannel(message) {
  ... // send message to the other side via the signaling channel
}

function receiveSignalingChannel (message) {
  // call this whenever we get a message on the signaling channel
  local.signalingChannel(message);
}

local.onaddstream = function (event) {
  // (videoElement is some <video> element)
  videoElement.src = URL.createObjectURL(event.stream);
};

DataChannel

The DataChannel interface represents a bi-directional data channel between two peers. A DataChannel is created via a factory method on a PeerConnection object. The corresponding DataChannel object is then dispatched at the other peer if the channel setup was successful.

Each DataChannel has an associated underlying data transport that is used to transport actual data to the other peer. The transport properties of the underlying data transport, such as reliability mode, is configured by the peer taking the initiative to create the channel. The receiving peer cannot change any transport properties of a offered data channel. The actual wire protocol between the peers is out of the scope for this specification.

A DataChannel created with createDataChannel() MUST initially be in the CONNECTING (0) state. If the DataChannel object’s underlying data transport is successfully set up, the user agent MUST announce the DataChannel as open.

When the user agent is to announce a DataChannel as open, the user agent MUST queue a task to run the following steps:

1. If the associated PeerConnection object’s PeerConnection readiness state is CLOSED (3), abort these steps.

2. Let channel be the DataChannel object to be announced.

3. Set channel’s readyState attribute to OPEN (1).

4. Fire a simple event named open at channel.

When an underlying data transport has been established, the user agent, of the peer that did not initiate the creation process MUST queue a task to run the following steps:

1. If the associated PeerConnection object’s PeerConnection readiness state is CLOSED (3), abort these steps.

2. Let configuration be an information bundle with key-value pairs, received from the other peer as a part of the process to establish the underlying data channel.

3. Let channel be a newly created DataChannel object.

4. Initialize channel’s label attribute to value that corresponds to the "label" key in configuration.

5. Initialize channel’s reliable attribute to true.

6. If configuration contains a key named "reliable", set channel’s reliable attribute to the corresponding value.

7. Set channel’s readyState attribute to OPEN (1).

8. Fire a datachannel event named datachannel with channel at the PeerConnection object.

When the process of tearing down a DataChannel object’s underlying data transport is initiated, the user agent MUST run the following steps:

1. If the associated PeerConnection object’s PeerConnection readiness state is CLOSED (3), abort these steps.

2. Let channel be the DataChannel object which is about to be closed.

3. If channel’s readyState is CLOSING (2) or CLOSED (3), then abort these steps.

4. Set channel’s readyState attribute to CLOSING (2).

5. Queue a task to run the following steps:

6. 1. Close channel’s underlying data transport.

      The data transport protocol will specify what happens to, e.g. buffered data, when the data transport is closed.

   2. Set channel’s readyState attribute to CLOSED (3).

   3. Fire a simple event named close at channel.

readonly attribute DOMString label

The DataChannel.label attribute represents a label that can be used to distinguish this DataChannel object from other DataChannel objects. The attribute MUST return the value to which it was set when the DataChannel object was created.

readonly attribute boolean reliable

The DataChannel.reliable attribute returns true if the DataChannel is reliable, and false otherwise. The attribute MUST return the value to which it was set when the DataChannel was created.

const unsigned short CONNECTING = 0

The user agent is attempting to establish the underlying data transport. This is the initial state of a DataChannel object created with createDataChannel().

const unsigned short OPEN = 1

The underlying data transport is established and communication is possible. This is the initial state of a DataChannel object dispatched as a part of a DataChannelEvent.

const unsigned short CLOSING = 2

The process of closing down the underlying data transport has started.

const unsigned short CLOSED = 3

The underlying data transport has been closed or could not be established.

readonly attribute unsigned short readyState

The DataChannel.readyState attribute represents the state of the DataChannel object. It MUST return the value to which the user agent last set it (as defined by the processing model algorithms). The attribute can have the following values: CONNECTING, OPEN, CLOSING or CLOSED.

readonly attribute unsigned long bufferedAmount

FIXME: align behavior with WebSocket API

[TreatNonCallableAsNull] attribute Function? onopen

This event handler, of type open, MUST be supported by all objects implementing the DataChannel interface.

[TreatNonCallableAsNull] attribute Function? onerror

This event handler, of type error, MUST be supported by all objects implementing the DataChannel interface.

[TreatNonCallableAsNull] attribute Function? onclose

This event handler, of type close, MUST be supported by all objects implementing the DataChannel interface.

void close()

Closes the DataChannel. It may be called regardless if the DataChannel object was created by this peer or the remote peer.

When the close() method is called, the user agent MUST initiate the process of tearing down the DataChannel object’s underlying data transport.

[TreatNonCallableAsNull] attribute Function? onmessage

This event handler, of type message, MUST be supported by all objects implementing the DataChannel interface.

attribute DOMString binaryType

FIXME: align behavior with WebSocket API

void send(DOMString data)

FIXME: align behavior with WebSocket API

void send(ArrayBuffer data)

FIXME: align behavior with WebSocket API

void send(Blob data)

FIXME: align behavior with WebSocket API

boolean reliable

-

Examples

var chan = peerConn.createDataChannel("mylabel");

chan.onmessage = function (evt) {
    // use evt.data
};

chan.send("hello");
    

peerConn.ondatachannel = function (evt) {
    var chan = evt.channel;

    chan.onmessage = function (evt) {
        // use evt.data
    };

    chan.onclose = function () {
        // remote side closed the data channel
    };
};
    

MediaStreamTrackEvent

The addtrack and removetrack events use the MediaStreamTrackEvent interface.

Firing a track event named e with a MediaStreamTrack track means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the MediaStreamTrackEvent interface with the track attribute set to track, MUST be created and dispatched at the given target.

readonly attribute MediaStreamTrack? track

The track attribute represents the MediaStreamTrack object associated with the event.

MediaStreamTrack? track

-

MediaStreamEvent

The addstream and removestream events use the MediaStreamEvent interface.

Firing a stream event named e with a MediaStream stream means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the MediaStreamEvent interface with the stream attribute set to stream, MUST be created and dispatched at the given target.

readonly attribute MediaStream? stream

The stream attribute represents the MediaStream object associated with the event.

MediaStream? stream

-

DataChannelEvent

The datachannel event use the DataChannelEvent interface.

Firing a datachannel event named e with a DataChannel channel means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the DataChannelEvent interface with the channel attribute set to channel, MUST be created and dispatched at the given target.

readonly attribute DataChannel? channel

The channel attribute represents the DataChannel object associated with the event.

DataChannel? channel

-

Constraints Registrations

IANA is requested to register the constraints defined in Constraints Section as specified in [[!RTCWEB-CONSTRAINTS]].

application/html-peer-connection-data

This registration is for community review and will be submitted to the IESG for review, approval, and registration with IANA.

Type name:

application

Subtype name:

html-peer-connection-data

Required parameters:

No REQUIRED parameters

Optional parameters:

No OPTIONAL parameters

Encoding considerations:

This MIME type defines a binary protocol format which uses UTF-8 for text encoding.

Security considerations:

This format is used for encoding UDP packets transmitted by potentially hostile Web page content via a trusted user agent to a destination selected by a potentially hostile remote server. To prevent this mechanism from being abused for cross-protocol attacks, all the data in these packets is masked so as to appear to be random noise. The intent of this masking is to reduce the potential attack scenarios to those already possible previously.

However, this feature still allows random data to be sent to destinations that might not normally have been able to receive them, such as to hosts within the victim’s intranet. If a service within such an intranet cannot handle receiving UDP packets containing random noise, it might be vulnerable to attack from this feature.

Interoperability considerations:

Rules for processing both conforming and non-conforming content are defined in this specification.

Published specification:

This document is the relevant specification.

Applications that use this media type:

This type is only intended for use with SDP. [[!SDP]]

Additional information:

Magic number(s):

No sequence of bytes can uniquely identify data in this format, as all data in this format is intentionally masked to avoid cross-protocol attacks.

File extension(s):

This format is not for use with files.

Macintosh file type code(s):

This format is not for use with files.

Person & email address to contact for further information:

Daniel C. Burnett <dburnett@voxeo.com>

Intended usage:

Common

Restrictions on usage:

No restrictions apply.

Author:

Daniel C. Burnett <dburnett@voxeo.com>

Change controller:

W3C

Fragment identifiers cannot be used with application/html-peer-connection-data as URLs cannot be used to identify streams that use this format.