When listing the applications the apps lock was incorrectly
locked twice instead of being locked and then unlocked.

ASTERISK-28302

Change-Id: If7d064592a9e88c0f1049214c50e02be6dabf79e

When processing SSRC attributes we were iterating through
all of them, even though we only need to know the remote
SSRC once. This was problematic because some browsers group
SSRCs together on a stream, and due to our negotiation only
end up using the first one. Since we set the second one as
the remote SSRC we would drop the received media from them
instead of allowing it through.

In the future this may be extended to allow SSRC groups
and to use information from the attributes.

Change-Id: I4dc87087dbe56a83aa65f0f897bbd4ca75ec1270

To prevent one subsystem's taskprocessors from causing others
to stall, new capabilities have been added to taskprocessors.

* Any taskprocessor name that has a '/' will have the part
  before the '/' saved as its "subsystem".
  Examples:
  "sorcery/acl-0000006a" and "sorcery/aor-00000019"
  will be grouped to subsystem "sorcery".
  "pjsip/distributor-00000025" and "pjsip/distributor-00000026"
  will bn grouped to subsystem "pjsip".
  Taskprocessors with no '/' have an empty subsystem.

* When a taskprocessor enters high-water alert status and it
  has a non-empty subsystem, the subsystem alert count will
  be incremented.

* When a taskprocessor leaves high-water alert status and it
  has a non-empty subsystem, the subsystem alert count will be
  decremented.

* A new api ast_taskprocessor_get_subsystem_alert() has been
  added that returns the number of taskprocessors in alert for
  the subsystem.

* A new CLI command "core show taskprocessor alerted subsystems"
  has been added.

* A new unit test was addded.

REMINDER: The taskprocessor code itself doesn't take any action
based on high-water alerts or overloading.  It's up to taskprocessor
users to check and take action themselves.  Currently only the pjsip
distributor does this.

* A new pjsip/global option "taskprocessor_overload_trigger"
  has been added that allows the user to select the trigger
  mechanism the distributor uses to pause accepting new requests.
  "none": Don't pause on any overload condition.
  "global": Pause on ANY taskprocessor overload (the default and
  current behavior)
  "pjsip_only": Pause only on pjsip taskprocessor overloads.

* The core pjsip pool was renamed from "SIP" to "pjsip" so it can
  be properly grouped into the "pjsip" subsystem.

* stasis taskprocessor names were changed to "stasis" as the
  subsystem.

* Sorcery core taskprocessor names were changed to "sorcery" to
  match the object taskprocessors.

Change-Id: I8c19068bb2fc26610a9f0b8624bdf577a04fcd56

Event type filtering is now enabled, and configurable per application. An app is
now able to specify which events are sent to the application by configuring an
allowed and/or disallowed list(s). This can be done by issuing the following:

PUT /applications/{applicationName}/eventFilter

And then enumerating the allowed/disallowed event types as a body parameter.

ASTERISK-28106

Change-Id: I9671ba1fcdb3b6c830b553d4c5365aed5d588d5b

p2p_write updates txformat but doesn't require a smoother.  If a smoother
was created by another bridge type the smoother could fall out of date causing
one way audio issues.  To prevent this the smoother is now destroyed on the
start of native bridge.

ASTERISK-28284 #close

Change-Id: I84e67f144963787fff9b4d79ac500514fb40cdc6

Currently, the Asterisk's pjsip_session module does not keeping the
rtcp's stats info after it was removed. But by adding the results
vector and keeping it until session is destroying, it can give more
useful information for other modules.

ASTERISK-28253

Change-Id: Ib25c2d3fc4da084aecfde2a82c1b1d733bd64fa5

A previous patch attempt to mitigate blocked threads on transport shutdown for
a given contact. It was thought that a second lock could be avoided by checking
the 'removing' flag on the transport monitor twice (once before and once after
the normal named aor locking). However as with usual threading issues if the
timing was right the original problem still occured.

This patch adds locking around the first 'removing' flag check and set, thus
nullifying the secondary check, so it was removed.

ASTERISK-28213

Change-Id: Iaa8e36e5311789549b76d8de42dfcea96013b2ed

When Asterisk is connected and used with a database the response
time of the database can cause problems in Asterisk if it is long.
Normally the only way to see this problem would be to retrieve a
backtrace from Asterisk and examine where things are blocked, or
examine the database to see if there is any indication of a
problem.

This change adds some basic query logging to make it easier to
investigate such a problem. When logging is enabled res_odbc will
now keep track of the number of queries executed, as well as the
query that has taken the longest time to execute. There is also
an option which will cause a WARNING message to be output if a
query takes longer than a configurable amount of time to execute.

This makes it easier and clearer for users that their database may
be experiencing a problem that could impact Asterisk.

ASTERISK-28277

Change-Id: I173cf4928b10754478a6a8c27dfa96ede0f058a6

At AstriCon, there was a strong desire for the ability to completely
bypass dialplan when using ARI. This is possible through the automatic
creation of a context and a couple of extensions whenever an application
is started.

For example, if you have an application named 'ari-example', a context
named 'stasis-ari-example' will be automatically created whenever this
application is started as long as one does not already exist. Two
extensions (a match-all extension for Stasis and a 'h' extension) are
created within this context. Any endpoint that registers to Asterisk
within this context will send all calls to the corresponding Stasis
application. When the application is destroyed, the context is removed.

ASTERISK-28104 #close

Change-Id: Ie35bd93075e05b05e3ae129a83c9426931b7ebac

Added ARI resource.
GET /ari/asterisk/ping : It returns "pong" message with timestamp
and asterisk id. It would be useful for simple heath check.

Change-Id: I8d24e1dcc96f60f73437c68d9463ed746f688b29

The context specified by 'regcontext' was not being created, so when Asterisk
attempted to later dynamically add an extension it would fail. This patch now
creates the context if a 'regcontext' is specified.

ASTERISK-28238

Change-Id: I0f36cf4ab0a93ff4b1cc5548d617ecfd45e09265

Testing revealed that the cache added no benefit but that it could
consume excessive memory.

Two new index related functions were created:
ast_sounds_get_index_for_file() and ast_media_index_update_for_file()
which restrict index updating to specific sound files.

The original ast_sounds_get_index() and ast_media_index_update()
calls are still available but since they no longer cache the results
internally, developers should re-use an index they may already have
instead of calling ast_sounds_get_index() repeatedly.  If information
for only a single file is needed, ast_sounds_get_index_for_file()
should be called instead of ast_sounds_get_index().

The media_index directory scan code was elimininated in favor of
using the existing ast_file_read_dirs() function.

Since there's no more cache, ast_sounds_index_init now only
registers the sounds cli commands instead of generating the
initial index and subscribing to stasis format register/unregister
messages.

"sounds" is no longer a valid target for the "module reload"
command.

Both the sounds cli commands and the sounds ari resources were
refactored to only call ast_sounds_get_index() once per invocation
and to use ast_sounds_get_index_for_file() when a specific sound
file is requested.

Change-Id: I1cef327ba1b0648d85d218b70ce469ad07f4aa8d

Control frames (PING / PONG / CLOSE) can be received in the middle of a
fragmented message. In order to ensure they do not interfere with the
reassembly buffer, we exit early and do not return the payload to the
caller.

ASTERISK-28257 #close

Change-Id: Ia5367144fe08ac6141bba3309517a48ec7f013bc

When a reliable transport is shutdown it's possible for the pjsip registrar
resource shutdown handler to get called multiple times. If this happens and one
of the threads is taking "too long" (slow database call for instance) then the
others get blocked waiting to delete.

Since it only takes one to delete the contact then the other threads should be
able to continue on if one of the threads is currently "deleting". This patch
makes it so now when a thread enters the shutdown handler it checks to see if a
thread is currently already "deleting". If so, then the thread does not attempt
to get the lock, and instead continues on thus avoiding the blockage.

ASTERISK-28213 #close

Change-Id: I7563ca596312b1dff4f3ab41483e89fe2862328a

To avoid the stream name collide if there're more than one video track
in one client. If client has multi video tracks, the name of ast_stream
which represents each video track may be the same. Use the MSID:LABEL
here because it's identifiable.

ASTERISK-28196 #close
Reported-by: xiemchen

Change-Id: Ib62b2886e8d3a30e481d94616b0ceaeab68a870b

This ensures that Asterisk responds properly to frames received from a
client with opcode 8 (CLOSE) by echoing back the status code in its own
CLOSE frame.

Handling of the CLOSE opcode is moved up with the rest of the opcodes so
that unmasking gets applied. The payload is no longer returned to the
caller, but neither ARI nor the chan_sip nor pjsip made use of the
payload, which is a good thing since it was masked.

ASTERISK-28231 #close

Change-Id: Icb1b60205fc77ee970ddc91d1f545671781344cf

The transport management code that checks for idle connections keeps a
reference to PJSIP's transport for IDLE_TIMEOUT milliseconds (32000 by
default). Because of this, if the transport is closed before this
timeout, the idle checking code will keep the transport from actually
being shutdown until the timeout expires.

Rather than passing the AO2 object to the scheduler task, we just pass
its key and look it up when it is time to potentially close the idle
connection. The other transport management code handles cleaning up
everything else for us.

Additionally, because we use the address of the transport when
generating its name, we concatenate an incrementing ID to the end of the
name to guarantee uniqueness.

Related to ASTERISK~28231

Change-Id: I02ee9f4073b6abca9169d30c47aa69b5e8ae9afb

Previously both AMI and ARI used a default route on
their stasis message router to handle some of the
messages for publishing out their respective
connection. This caused messages to be given to
their subscription that could not be formatted
into AMI or JSON.

This change adds an API call to the stasis message
router which allows a default route to be set as well
as formatters that the default route is expecting.
This allows both AMI and ARI to specify that their
default route only wants messages of their given
formatter. By doing so stasis can more intelligently
filter at publishing time so that they do not receive
messages which will not be turned into AMI or JSON.

ASTERISK-28244

Change-Id: I65272819a53ce99f869181d1d370da559a7d1703

When --enable-dev-mode is used, pjsip_tpmgr_receive_packet() will assert
if passed a payload length of 0, so treat empty frames as if we didn't
receive them.

Change-Id: I9c5fdccd89cc8d2f3ed7e3ee405ef0fc78178f48

The commit I2f97ebfa79969a36a97bb7b9afd5b6268cf1a07d removed sending out
the ContactStatus AMI event when a contact is updated.
Thist change broke things which rely on old behavior.

This patch adds a new PJSIP global configuration option
'send_contact_status_on_update_registration' to be able to preserve old
ContactStatus behavior.
By default new behavior, i.e. the ContactStatus event will not be sent when a
device refreshes its registration.

Change-Id: I706adf7584e7077eb6bde6d9799ca408bc82ce46

For video streams it was possible for the abs-send-time information
to be placed into RTP streams even if not negotiated. Depending on
the endpoint in use this could cause video to not flow.

We now only enable abs-send-time for negotiation if WebRTC is enabled.

ASTERISK-28230

Change-Id: I0eb682302f8da3a4ea3c42e839208d55f825ed0c

The remote side may start a new stream when renegotiating RTP.
Need to reset the DTMF last sequence number and the timestamp
of the last END packet on RTP renegotiation.

If the new time stamp is lower then the timestamp of the last DTMF END packet
the asterisk drops all DTMF frames as out of order.

This bug was caught using Cisco ip-phone SPA5XX and codec g722.
On SIP session update the SPA50X resets stream and a new timestamp is twice
smaller then the previous.

ASTERISK-28162 #close

Change-Id: Ic72b4497e74d801b27a635559c1cf29c16c95254

All of the fields that were removed were no longer referenced except for
'lastrxts' and 'rxseqno' which were only ever written to.

Change-Id: I5a5d31eb33e97663843698f58d0d97f22a76627c

The profile-iop octet (the 2nd) of profile-level-id can be zero
according to RFC 6184 Section 8.1. So we ignore its value when deciding
to include profile-level-id in the outgoing SDP.

ASTERISK-27959 #close
Reported by: David Kuehling

Change-Id: Id28cd916a3d7748058fe9609b585d07d9e243f73

Change-Id: I050ceffe5a133d5add2dab46687209813d58f597

This reverts commit 3f530412.

Pending resolution of ASTERISK_28200

Change-Id: Iad4f3614cac95b00fdbb2b799aab8ae6285ec988

The ARI DELETE /channels command takes a "reason" parameter
Previously, there were only five reasons implemented
This patch adds more reasons to choose from for more
complex setups

ASTERISK-28198 #close

Change-Id: I85996f1076c9946d65c778413f040a845a90fecc

The session_supplements for the pjsip makes crashes when the module
load/unload.

ASTERISK-28157

Change-Id: I5b82be3a75d702cf1933d8d1417f44aa10ad1029

* The bridging core no longer uses the stasis cache for bridge
  snapshots.  The latest bridge snapshot is now stored on the
  ast_bridge structure itself.

* The following APIs are no longer available since the stasis cache
  is no longer used:
    ast_bridge_topic_cached()
    ast_bridge_topic_all_cached()

* A topic pool is now used for individual bridge topics.

* The ast_bridge_cache() function was removed since there's no
  longer a separate container of snapshots.

* A new function "ast_bridges()" was created to retrieve the
  container of all bridges.  Users formerly calling
  ast_bridge_cache() can use the new function to iterate over
  bridges and retrieve the latest snapshot directly from the
  bridge.

* The ast_bridge_snapshot_get_latest() function was renamed to
  ast_bridge_get_snapshot_by_uniqueid().

* A new function "ast_bridge_get_snapshot()" was created to retrieve
  the bridge snapshot directly from the bridge structure.

* The ast_bridge_topic_all() function now returns a normal topic
  not a cached one so you can't use stasis cache functions on it
  either.

* The ast_bridge_snapshot_type() stasis message now has the
  ast_bridge_snapshot_update structure as it's data.  It contains
  the last snapshot and the new one.

* cdr, cel, manager and ari have been updated to use the new
  arrangement.

Change-Id: I7049b80efa88676ce5c4666f818fa18ad1985369

When a channel snapshot was created it used to be done
from scratch, copying all data (many strings). This incurs
a cost when doing so.

This change segments the channel snapshot into different
components which can be reused if unchanged from the
previous snapshot creation, reducing the cost. In normal
cases this results in some pointers being copied with
reference count being bumped, some integers being set,
and a string or two copied. The other benefit is that it
is now possible to determine if a channel snapshot update
is redundant and thus stop it before a message is published
to stasis.

The specific segments in the channel snapshot were split up
based on whether they are changed together, how often they
are changed, and their general grouping. In practice only
1 (or 0) of the segments actually get changed in normal
operation.

Invalidation is done by setting a flag on the channel when
the segment source is changed, forcing creation of a new
segment when the channel snapshot is created.

ASTERISK-28119

Change-Id: I5d7ef3df963a88ac47bc187d73c5225c315f8423

Channels no longer use the Stasis cache for channel snapshots. Instead
they are stored in a hash table in stasis_channels which reduces the
number of Stasis messages created and allows better storage.

As a result the following APIs are no longer available since the stasis
cache is no longer used:
ast_channel_topic_cached()
ast_channel_topic_all_cached()

The ast_channel_cache_all() and ast_channel_cache_by_name() functions
now return an ao2_container of ast_channel_snapshots rather than
a container of stasis_messages therefore you can't (and don't need
to) call stasis_cache functions on it.

The ast_channel_topic_all() function now returns a normal topic not
a cached one so you can't use stasis cache functions on it either.

The ast_channel_snapshot_type() stasis message now has the
ast_channel_snapshot_update structure as it's data. It contains the
last snapshot and the new one.

ast_channel_snapshot_get_latest() still returns the latest snapshot.

The latest snapshot is now stored on the channel itself to eliminate
cache hits when Stasis messages that have the snapshot as a payload
are created.

ASTERISK-28102

Change-Id: I9334febff60a82d7c39703e49059fa3a68825786

The marker bit set on the voice packet indicates the start
of a new stream and a new time stamp.
Need to reset the DTMF last sequence number and the timestamp
of the last END packet.

If the new time stamp is lower then the timestamp of the last DTMF END packet
the asterisk drops all DTMF frames as out of order.

This bug was caught using Cisco ip-phone SPA50X and codec g722.
On SIP session update the SPA50X resets stream indicating it with market bit
and a new timestamp is twice smaller then the previous.

ASTERISK-28162 #close

Change-Id: If9c5742158fa836ad549713a9814d46a5d2b1620

Replace usage of ao2_container_alloc with ao2_container_alloc_hash or
ao2_container_alloc_list.  Remove ao2_container_alloc macro.

Change-Id: I0907d78bc66efc775672df37c8faad00f2f6c088

This change adds the ability for subscriptions to indicate
which message types they are interested in accepting. By
doing so the filtering is done before being dispatched
to the subscriber, reducing the amount of work that has
to be done.

This is optional and if a subscriber does not add
message types they wish to accept and set the subscription
to selective filtering the previous behavior is preserved
and they receive all messages.

There is also the ability to explicitly force the reception
of all messages for cases such as AMI or ARI where a large
number of messages are expected that are then generically
converted into a different format.

ASTERISK-28103

Change-Id: I99bee23895baa0a117985d51683f7963b77aa190

The res_ari(POST /channels/create handler) deos not check the endpoint
parameter length. And it causes core
dump.
Fixed it to check the parameter length. Also fixed memory leak.

ASTERISK-28169

Change-Id: Ibf10a9eb8a2e3a9ee1e13fbe748b2ecf955c3993

PJSIP assumes that these header names are not allocated, does not clone
the name strings when reusing headers.

Block unload of res_pjsip_caller_id until shutdown to ensure static
memory stays valid.  It was previously unsafe to unload while any
sessions are active.

Change-Id: I190854dea943d6e441cf03733f8a0da661aea27f

The presence of Record-Route in re-invites is optional, thus it is
important to make sure the dialog doesn't have a routset before
rewriting the contact header.

ASTERISK-28129 #close

Change-Id: Ic8ceb54ccfc93f7e315e476c514a2c777f2da7dc

When Asterisk's taskprocessors get overloaded we need to reduce the work
load. res_pjsip currently ignores new SIP requests and relies on SIP
retransmissions in the hope that the overload condition will clear soon
enough to handle the retransmitted SIP request.
This change adds the following code after ast_taskprocessor_alert_get()
has returned TRUE:
1- identifies transport type. If non-udp then send a 503 response
2- if transport type is udp/udp6 then ignore, as before.

Change-Id: I1c230b40d43a254ea0f226b7acf9ee480a5d3836

The use of a '|' in the "global/debug" synopsis documentation caused the
generated html table on the wiki to add an extra column that included the
text after the pipe.

This patch replaces the pipe with a comma.

ASTERISK-28150

Change-Id: I3d79a6ca6d733d9cb290e779438114884b98a719

The current round-robin method does not take the current taskprocessor
load into consideration when distributing requests.  Using the least-size
method the request goes to the taskprocessor that is servicing the least
number of active tasks at the current time.

Longer running tasks with the round-robin method can delay processing
tasks.

* Change the algorithm from round-robin to least-size for picking the
PJSIP taskprocessor from the default serializer pool.

Change-Id: I7b8d8cc2c2490494f579374b6af0a4868e3a37cd