Resolves an edge case dtls negotiation delay for certain networks which
somehow manage to drop the rtcp side's packet when these are both sent
ast_rtp_remote_address_set, causing it to have to time-out and restart
the handshake.

Move dtls pending bio flush in to it's own function, and call it from
ast_rtp_on_ice_complete, when we're rtp->ice, rather than when
ast_rtp_remote_address_set.

Keep the existing flush from the recent change to res_rtp_remote_address_set
if ice is not being used.

ASTERISK-25614 #close
Reported-by: XenCALL
Tested by: XenCALL

Change-Id: Ie2caedbdee1783159f375589b6fd3845c8577ba5

This change introduces the configuration option 'full_backend_cache'
which changes the cache to be a full mirror of the backend instead
of a per-object cache. This allows all sorcery retrieval operations
to be carried out against it and is useful for object types which
are used in a "retrieve all" or "retrieve some" pattern.

ASTERISK-25625 #close

Change-Id: Ie2993487e9c19de563413ad5561c7403b48caab5

When applying an empty DTLS configuration the filenames in the
configuration will be empty. This is actually valid to do and
each filename should simply be ignored.

Change-Id: Ib761dc235638a3fb701df337952f831fc3e69539

Per the documentation the WebSocket support in chan_sip is
supposed to be enabled by default but is not. This change
corrects that.

Change-Id: Icb02bbcad47b11a795c14ce20a9bf29649a54423

The JSON library Asterisk uses, jansson, is not thread
safe for us in a few ways. To help with this wrappers for JSON
object reference count increasing and decreasing were added
which use a global lock to ensure they don't clobber over
each other. This does not extend to reference count manipulation
within the jansson library itself. This means you can't safely
use the object borrowing specifier (O) in ast_json_pack and
you can't share JSON instances between objects.

This change removes uses of the O specifier and replaces them
with the o specifier and an explicit ast_json_ref. Some cases
of instance sharing have also been removed.

ASTERISK-25601 #close

Change-Id: I06550d8b0cc1bfeb56cab580a4e608ae4f1ec7d1

- Trigger pending DTLS packets to send out, once the RTP instance's remote
  address is set.
- Avoids locking the DTLS structure unnecessarily by only doing this if
  DTLS is passive.
- Add DTLS locks around the structurally sensitive calls in the SSL
  portion of __rtp_recvfrom, since dtls_srtp_check_pending does not lock
  inside of itself, and we're dealing with the SSL BIO in at least two
  threads.

WebRTC channels may receive a DTLS handshake before
ast_rtp_remote_address_set is called, which causes there to be a pending
response to send out.   Previous to 1ad827, this was handled by calling
dtls_srtp_check_pending on receipt of any RTP packet - a STUN or RTP
packet could trigger the pending handshake response.  Since that was
rightfully removed, whenever the DTLS handshake is received before the
remote address is set, we would have to wait until another SSL packet
arrives.

As of Chrome M47's optimizations to their handshake process, WebRTC
conversations between Chrome M47+ and Asterisk, where Asterisk is passive,
experience a 1 second delay without this patch, because the SSL handshake
is received before ICE negotation stores the remote_address, and the next
SSL packet isn't received until after a 1 second timeout in Chrome, which
causes a new handshake request.

ASTERISK-25614 #close

Change-Id: I547f1be7e302dbf71f6553dd8cbc0657b1d0b908

When the asterisk sending OriginateResponse message,
it doesn't set the "Uniqueid".
And it didn't support correct response message for
Application originate.

ASTERISK-25624 #close

Change-Id: I26f54f677ccfb0b7cfd4967a844a1657fd69b74d

ASTERISK-25615
Reported by: George Joseph

Change-Id: Ieff35307254ca193f3d473cff2e396ca57c7ce0b

If a call enters on a queue and the members on that queue are updated in
realtime (ex: using mysql inserting a new agent) the queue members are
never refreshed and the call will stay in the queue until other event occurs.
This happens only if this is the first call of the queue and there is no
agents servicing.
This patch prevent this issue, ensuring realtime members are updated if
there is one call in the queue and no available agents

ASTERISK-25442 #close

Change-Id: If1e036d013a5c1d8b0bf60d71d48fe98694a8682

An ERROR or WARNING message should generally indicate that something has gone
wrong in Asterisk. In the case of writing to a file descriptor, Asterisk is not
in control of when the far end closes its reading on a file descriptor. If the
far end does close the file descriptor in an unclean fashion, this isn't a bug
or error in Asterisk, particularly when the situation can be gracefully
handled in Asterisk.

Currently, when this happens, a user would see the following somewhat cryptic
ERROR message:

  "utils.c: write() returned error: Broken pipe"

There's a few problems with this:
(1) It doesn't provide any context, other than 'something broke a pipe'
(2) As noted, it isn't actually an error in Asterisk
(3) It can get rather spammy if the thing breaking the pipe occurs often, such
    as a FastAGI server
(4) Spammy ERROR messages make Asterisk appear to be having issues, or can even
    mask legitimate issues

This patch changes ast_carefulwrite to only log an ERROR if we actually had one
that was reasonably under our control. For debugging purposes, we still emit
a debug message if we detect that the far side has stopped reading.

Change-Id: Ia503bb1efcec685fa6f3017bedf98061f8e1b566

pjproject < 2.5.0 will segfault on a tls transport if async_operations
is greater than 1.  A runtime version check has been added to throw
an error if the version is < 2.5.0 and async_operations > 1.

To assist in the check, a new api "ast_compare_versions" was added
to utils which compares 2 major.minor.patch.extra version strings.

ASTERISK-25615 #close

Change-Id: I8e88bb49cbcfbca88d9de705496d6f6a8c938a98
Reported-by: George Joseph
Tested-by: George Joseph

Adds the TCP Keep Alive option to TCP and TLS server sockets. Previously
this option was only being set on session sockets.
http://www.tldp.org/HOWTO/html_single/TCP-Keepalive-HOWTO/
According to the link above, the SO_KEEPALIVE option is useful for knowing
when a TCP connected endpoint has severed communication without indicating
it or has become unreachable for some reason. Without this patch, keep
alive is not set on the socket listening for incoming TCP sessions and
in Komatsu's report this resulted in the thread listening for TCP becoming
stuck in a waiting state.

ASTERISK-25364 #close
Reported by: Hiroaki Komatsu

Change-Id: I7ed7bcfa982b367dc64b4b73fbd962da49b9af36

The default value was never set for audio_buffers, causing bad
audio quality.  This ensures the default is always set.

ASTERISK-25569 #close

Change-Id: I2d2ee3e644120b0f9f6ea6ab9286d7d590942a44

Fixed a bug that originally would show a negative number of
active calls occuring in Asterisk. A gauge is persistent so
incrementing and decrementing it results in a more consistent
performance. Also changed to the call to StatsD to use
ast_statsd_log_string() so that a "+" could be sent to StatsD.

ASTERISK-25619 #close

Change-Id: Iaaeff5c4c6a46535366b4d16ea0ed0ee75ab2ee7

Both transport and endpoint now check for the existence and readability
of tls certificate and key files before passing them on to pjproject.
This will cause the object to not load rather than waiting for pjproject
to discover that there's a problem when a session is attempted.

NOTE: chan_sip also uses ast_rtp_dtls_cfg_parse but it's located
in build_peer which is gigantic and I didn't want to disturb it.
Error messages will emit but it won't interrupt chan_sip loading.

ASTERISK-25618 #close

Change-Id: Ie43f2c1d653ac1fda6a6f6faecb7c2ebadaf47c9
Reported-by: George Joseph
Tested-by: George Joseph

The current logic for ICE negotiation starts it
when receiving an SDP with ICE candidates. This is
incorrect as ICE negotiation can only start when each 
call party have at least one pair of local and remote 
candidate. Starting ICE negotiation early would result 
in negotiation failure and ultimately no audio.

This change makes it so ICE negotiation is only started
when a response with SDP is received or when a response
with SDP is sent.

ASTERISK-24146

Change-Id: I55a632bde9e9827871b09141d82747e08379a8ca

Asterisk may crash when calling ast_channel_get_t38_state(c)
on a locked channel which is being hung up.

ASTERISK-25609 #close

Change-Id: Ifaa707c04b865a290ffab719bd2e5c48ff667c7b

See ASTERISK-25615.
If the transport protocol is tls and async_operations > 1, pjproject
will segfault if more than one operation is attempted on the same socket.
Until this is fixed upstream, a check has been added to throw an error
if a tls transport config has async_operations set to > 1.

ASTERISK-25615

Change-Id: I76b9a5b2a5a0054fe71ca5851e635f2dca7685a6
Reported-by: George Joseph
Tested-by: George Joseph

ASTERISK-25599 #close

Change-Id: Idbd187f711b2ec63dda949ca0f79aa0c1a0a0b6e

ASTERISK-25616 #close

Change-Id: Ibe729aaf2e6e25506cff247cec5149ec1e589319

A crash happens sometimes when performing a CLI "sip reload".  The bogus
peer gets refreshed while it is in use by a new call which can cause the
crash.

* Protected the global bogus peer object with an ao2 global object
container.

ASTERISK-25610 #close

Change-Id: I5b528c742195681abcf713c6e1011ea65354eeed

Current support for reason header did work only in SIP responses.
According to RFC3336 the reason header might appear in any SIP request.
But it seems to make most sence in BYE and CANCEL so parasing is done
there too (if use_q850_reason=yes).

Change-Id: Ib6be7b34c23a76d0e98dfd0816c89931000ac790

This reverts commit f42d22d3.

Unfortunately, using a bridge to manage T.38 state will cause severe deadlocks
in core_unreal/chan_local. Local channels attempt to reach across both their
peer and the peer's bridge to inspect T.38 state. Given the propensity of
Local channel chains, managing the locking situation in such a scenario is
practically infeasible.

Change-Id: I932107387c13aad2c75a7a4c1e94197a9d6d8a51

It will never be perfect or even pretty, mostly because of the differences
between static and dynamic contacts.

Created:

Can't use the contact or contact_status alloc functions
because the objects come and go regardless of the actual state.

Can't use the contact_apply_handler, ast_sip_location_add_contact or
a sorcery created handler because they only get called for dynamic
contacts.  Similarly, permanent_uri_handler only gets called for
static contacts.

So, Matt had it right. :)  ast_res_pjsip_find_or_create_contact_status is
the only place it can go and not have duplicated code.  Both
permanent_uri_handler and contact_apply_handler call find_or_create.

Removed:

Can't use the destructors for the same reason as above.  The only
place to put this is in persistent_endpoint_contact_deleted_observer
which I believe is the "correct" place but even that will handle only
dynamic contacts.  This doesn't called on shutdown however.  There is
no hook to use for static contacts that may be removed because of a
config change while asterisk is in operation.

I moved the cleanup of contact_status from ast_sip_location_delete_contact
to the handler as well.

Status Change and RTT:

Although they worked fine where they were (in update_contact_status) I
moved them to persistent_endpoint_contact_status_observer to make it
more consistent with removed.  There was logic there already to detect
a state change.

Finally, fixed a nit in permanent_uri_handler rmudgett reported
eralier.

ASTERISK-25608 #close

Change-Id: I4b56e7dfc3be3baaaf6f1eac5b2068a0b79e357d
Reported-by: George Joseph
Tested-by: George Joseph

ASTERISK-25584 #close

Change-Id: Iae00071b4ff1ae76f24995aeac4d00284fd14f91