This patch adds a new module, res_pjsip_history, that provides a slightly
better way of debugging SIP message traffic on a busy Asterisk system. The
existing mechanisms all rely on passively dumping a SIP message to the CLI.
While this is perfectly fine for logging purposes and well controlled
environments, on many installations, the amount of SIP messages Asterisk
receives will quickly swamp the CLI. This makes it difficult to view/capture
those messages that you want to diagnose in real time.

This patch provides another way of handling this. When enabled, the module
will store SIP message traffic in memory. This traffic can then be queried
at leisure.

In order to make the querying useful, a CLI command has been implemented,
'pjsip show history', that supports a basic expression syntax similar to
SQL or other query languages. A small number of useful fields have been
added in this initial patch; additional fields can easily be added in
later improvements. Those fields are:
 - number: The entry index in the history
 - timestamp: The time the message was recieved
 - addr: The source/destination address of the message
 - sip.msg.request.method: The request method
 - sip.msg.call-id: The Call-ID header

Note - this is a resurrection of the module initially proposed on Review Board
here: https://reviewboard.asterisk.org/r/4053/

Change-Id: I39bd74ce998e99ad5ebc0aab3e84df3a150f8e36

The test_timezone_watch unit test is written to expect a
condition to be signaled when the inotify daemon thread runs.
There exists a small window where the test_timezone_watch
thread can signal the inotify daemon thread while it is not
reading on the underlying file descriptor. If this occurs
the test_timezone_watch thread will wait indefinitely for a
signal that will never arrive.

This change adds a timeout to the condition so it will return
regardless after a period of time.

Change-Id: Ifed981879df6de3d93acd3ee0a70f92546517390

The cache_clear test was written to expect duplicate Stasis messages
sent from the technology endpoint to the all caching topic. This patch
fixes the test to no longer expect these duplicate messages.

ASTERISK-25137

Change-Id: I58075d70d6cdf42e792e0fb63ba624720bfce981

When an endpoint is created, its messages are forwarded to both the tech
endpoint topic and the all endpoints topic. This is done so that various
parties interested in endpoint messages can subscribe to just the tech
endpoint and receive all messages associated with that particular technology,
as opposed to subscribing to the all endpoints topic. Unfortunately, when the
tech endpoint is created, it also forwards all of its messages to the all
topic. This results in duplicate messages whenever an endpoint publishes its
messages.

This patch resolves the duplicate message issue by creating a new function
for Stasis caching topics, stasis_cp_sink_create. In most respects, this acts
as a normal caching topic, save that it no longer forwards messages it receives
to the all endpoints topic. This allows it to act as an aggregation "sink",
while preserving the necessary caching behaviour.

ASTERISK-25137 #close
Reported-by: Vitezslav Novy

ASTERISK-25116 #close
Reported-by: George Joseph <george.joseph@fairview5.com>
Tested-by: George Joseph <george.joseph@fairview5.com>

Change-Id: Ie47784adfb973ab0063e59fc18f390d7dd26d17b

Change-Id: I9d88fac0394d5bbaff0900a2ee911c4e4478846b

websocket_write_timeout was not being set to its default value
during sip config reload, which meant that prior to this commit,
1) the default value of 100 was not used, unless an invalid value
(or 1) was specified in sip.conf for websocket_write_timeout, and
2) if the websocket_write_timeout directive was removed from sip.conf
without a full restart of asterisk, then the previous value would
continue to be used indefinitely.

This essentially lead to a 0ms write timeout (the first write attempt
in ast_careful_fwrite must have succeeded) in websocket write requests
from chan_sip, unless websocket_write_timeout was explicitely set in sip.conf.

Changes to websocket_write_timeout still only apply to new websocket
sessions, after the sip reload -- timeouts on existing sessions are
not adjusted during sip reload.

Change-Id: Ibed3816ed29cc354af6564c5ab3e75eab72cb953

Use of GOTO_ON_BLINDXFR would not work at all.  The target location would
never be executed by the transferring channel.

* Made feature_blind_transfer() call ast_bridge_set_after_go_on() with
valid context, exten, and priority parameters from the transferring
channel.

* Renamed some feature_blind_transfer() local variables for clarity.

ASTERISK-25641 #close
Reported by Dmitry Melekhov

Change-Id: I19bead9ffdc4aee8d58c654ca05a198da1e4b7ac

In 450579e9, a change was made that removed the deletion of the
'contact_status' object when a 'contact' object is deleted in sorcery.
This unfortunately means that the 'contact_status' object persists, even when
something has explicitly removed a contact. The result is that the state of
the contact will not be regenerated if that contact is re-created, and the
stale state will be reported/used for that contact. It also results in
no ContactStatusChanged events being generated for either ARI or AMI.

This patch restores the deletion logic that was removed. Doing so now
results in the expected events being generated again.

Change-Id: I28789a112e845072308b5b34522690e3faf58f07

Change-Id: I19b49112e1b630bd04e859f14ccf96f8ebd6b151

- The maximum_number_of_words was previously documented as being
the number of words that when exceeded, would result in the AMD
application returning that the audio represents a machine.

This was inconsistent with its actual functionality - it was
a number of words that when REACHED, would result in determination
as a machine.

This update corrects the functionality to match the previously
documented functionality.  This is a backwards incompatible change
in configuration file, and has been added to UPGRADE.txt as a result.

The sample configuration file and application defaults have been updated
so that the default value is now 2, which reflects the same default
functionality as previous versions.

- Update documentation for silence_threshold, which previously implied
that it was measuring time, rather than noise averages in the sample.

- Update the comments in amd.conf.sample.

ASTERISK-25639 #close
Change-Id: I4b1451e5dc9cb3cb06d59b6ab872f5275ba79093

Resolves an edge case dtls negotiation delay for certain networks which
somehow manage to drop the rtcp side's packet when these are both sent
ast_rtp_remote_address_set, causing it to have to time-out and restart
the handshake.

Move dtls pending bio flush in to it's own function, and call it from
ast_rtp_on_ice_complete, when we're rtp->ice, rather than when
ast_rtp_remote_address_set.

Keep the existing flush from the recent change to res_rtp_remote_address_set
if ice is not being used.

ASTERISK-25614 #close
Reported-by: XenCALL
Tested by: XenCALL

Change-Id: Ie2caedbdee1783159f375589b6fd3845c8577ba5

This change introduces the configuration option 'full_backend_cache'
which changes the cache to be a full mirror of the backend instead
of a per-object cache. This allows all sorcery retrieval operations
to be carried out against it and is useful for object types which
are used in a "retrieve all" or "retrieve some" pattern.

ASTERISK-25625 #close

Change-Id: Ie2993487e9c19de563413ad5561c7403b48caab5

When applying an empty DTLS configuration the filenames in the
configuration will be empty. This is actually valid to do and
each filename should simply be ignored.

Change-Id: Ib761dc235638a3fb701df337952f831fc3e69539

Per the documentation the WebSocket support in chan_sip is
supposed to be enabled by default but is not. This change
corrects that.

Change-Id: Icb02bbcad47b11a795c14ce20a9bf29649a54423

The JSON library Asterisk uses, jansson, is not thread
safe for us in a few ways. To help with this wrappers for JSON
object reference count increasing and decreasing were added
which use a global lock to ensure they don't clobber over
each other. This does not extend to reference count manipulation
within the jansson library itself. This means you can't safely
use the object borrowing specifier (O) in ast_json_pack and
you can't share JSON instances between objects.

This change removes uses of the O specifier and replaces them
with the o specifier and an explicit ast_json_ref. Some cases
of instance sharing have also been removed.

ASTERISK-25601 #close

Change-Id: I06550d8b0cc1bfeb56cab580a4e608ae4f1ec7d1

- Trigger pending DTLS packets to send out, once the RTP instance's remote
  address is set.
- Avoids locking the DTLS structure unnecessarily by only doing this if
  DTLS is passive.
- Add DTLS locks around the structurally sensitive calls in the SSL
  portion of __rtp_recvfrom, since dtls_srtp_check_pending does not lock
  inside of itself, and we're dealing with the SSL BIO in at least two
  threads.

WebRTC channels may receive a DTLS handshake before
ast_rtp_remote_address_set is called, which causes there to be a pending
response to send out.   Previous to 1ad827, this was handled by calling
dtls_srtp_check_pending on receipt of any RTP packet - a STUN or RTP
packet could trigger the pending handshake response.  Since that was
rightfully removed, whenever the DTLS handshake is received before the
remote address is set, we would have to wait until another SSL packet
arrives.

As of Chrome M47's optimizations to their handshake process, WebRTC
conversations between Chrome M47+ and Asterisk, where Asterisk is passive,
experience a 1 second delay without this patch, because the SSL handshake
is received before ICE negotation stores the remote_address, and the next
SSL packet isn't received until after a 1 second timeout in Chrome, which
causes a new handshake request.

ASTERISK-25614 #close

Change-Id: I547f1be7e302dbf71f6553dd8cbc0657b1d0b908

When the asterisk sending OriginateResponse message,
it doesn't set the "Uniqueid".
And it didn't support correct response message for
Application originate.

ASTERISK-25624 #close

Change-Id: I26f54f677ccfb0b7cfd4967a844a1657fd69b74d

ASTERISK-25615
Reported by: George Joseph

Change-Id: Ieff35307254ca193f3d473cff2e396ca57c7ce0b

If a call enters on a queue and the members on that queue are updated in
realtime (ex: using mysql inserting a new agent) the queue members are
never refreshed and the call will stay in the queue until other event occurs.
This happens only if this is the first call of the queue and there is no
agents servicing.
This patch prevent this issue, ensuring realtime members are updated if
there is one call in the queue and no available agents

ASTERISK-25442 #close

Change-Id: If1e036d013a5c1d8b0bf60d71d48fe98694a8682

An ERROR or WARNING message should generally indicate that something has gone
wrong in Asterisk. In the case of writing to a file descriptor, Asterisk is not
in control of when the far end closes its reading on a file descriptor. If the
far end does close the file descriptor in an unclean fashion, this isn't a bug
or error in Asterisk, particularly when the situation can be gracefully
handled in Asterisk.

Currently, when this happens, a user would see the following somewhat cryptic
ERROR message:

  "utils.c: write() returned error: Broken pipe"

There's a few problems with this:
(1) It doesn't provide any context, other than 'something broke a pipe'
(2) As noted, it isn't actually an error in Asterisk
(3) It can get rather spammy if the thing breaking the pipe occurs often, such
    as a FastAGI server
(4) Spammy ERROR messages make Asterisk appear to be having issues, or can even
    mask legitimate issues

This patch changes ast_carefulwrite to only log an ERROR if we actually had one
that was reasonably under our control. For debugging purposes, we still emit
a debug message if we detect that the far side has stopped reading.

Change-Id: Ia503bb1efcec685fa6f3017bedf98061f8e1b566

pjproject < 2.5.0 will segfault on a tls transport if async_operations
is greater than 1.  A runtime version check has been added to throw
an error if the version is < 2.5.0 and async_operations > 1.

To assist in the check, a new api "ast_compare_versions" was added
to utils which compares 2 major.minor.patch.extra version strings.

ASTERISK-25615 #close

Change-Id: I8e88bb49cbcfbca88d9de705496d6f6a8c938a98
Reported-by: George Joseph
Tested-by: George Joseph

Adds the TCP Keep Alive option to TCP and TLS server sockets. Previously
this option was only being set on session sockets.
http://www.tldp.org/HOWTO/html_single/TCP-Keepalive-HOWTO/
According to the link above, the SO_KEEPALIVE option is useful for knowing
when a TCP connected endpoint has severed communication without indicating
it or has become unreachable for some reason. Without this patch, keep
alive is not set on the socket listening for incoming TCP sessions and
in Komatsu's report this resulted in the thread listening for TCP becoming
stuck in a waiting state.

ASTERISK-25364 #close
Reported by: Hiroaki Komatsu

Change-Id: I7ed7bcfa982b367dc64b4b73fbd962da49b9af36

The default value was never set for audio_buffers, causing bad
audio quality.  This ensures the default is always set.

ASTERISK-25569 #close

Change-Id: I2d2ee3e644120b0f9f6ea6ab9286d7d590942a44

Fixed a bug that originally would show a negative number of
active calls occuring in Asterisk. A gauge is persistent so
incrementing and decrementing it results in a more consistent
performance. Also changed to the call to StatsD to use
ast_statsd_log_string() so that a "+" could be sent to StatsD.

ASTERISK-25619 #close

Change-Id: Iaaeff5c4c6a46535366b4d16ea0ed0ee75ab2ee7

Both transport and endpoint now check for the existence and readability
of tls certificate and key files before passing them on to pjproject.
This will cause the object to not load rather than waiting for pjproject
to discover that there's a problem when a session is attempted.

NOTE: chan_sip also uses ast_rtp_dtls_cfg_parse but it's located
in build_peer which is gigantic and I didn't want to disturb it.
Error messages will emit but it won't interrupt chan_sip loading.

ASTERISK-25618 #close

Change-Id: Ie43f2c1d653ac1fda6a6f6faecb7c2ebadaf47c9
Reported-by: George Joseph
Tested-by: George Joseph