During stress testing, we have frequently seen crashes occur because a
CLI or AMI command attempts to access information that is in the process
of being destroyed.

When addressing how to fix this issue, we initially considered fixing
individual crashes we observed. However, the changes required to fix
those problems would introduce considerable overhead to the nominal
case. This is not reasonable in order to prevent a crash from occurring
while Asterisk is already shutting down.

Instead, this change makes it so AMI and CLI commands cannot be executed
if Asterisk is being shut down. For AMI, this is absolute. For CLI,
though, certain commands can be registered so that they may be run
during Asterisk shutdown.

ASTERISK-25825 #close

Change-Id: I8887e215ac352fadf7f4c1e082da9089b1421990

The configuration unsigned integer option handler sets flags for the
parser as if the option should be a signed integer (PARSE_INT32),
leading to errors on "out of range" values. Fix flags (PARSE_UINT32).

A fix to res_pjsip is also present which stops invalid flags from
being passed when registering sorcery object fields for qualify
status.

ASTERISK-25612 #close

Change-Id: I96b539336275e0e72a8e8033487d2c3344debd3e

The log message when a MusicOnHold music class was not found was changed
from debug level to WARNING level in Asterisk 11.19 and 13.5.  For those
using realtime musiconhold, this message is wrong because it warns
before checking the database.

This changeset delays the warning until after the database has been
checked.

Reported-by: Conrad de Wet
ASTERISK-25444 #close

Change-Id: I6cfb2db2f9cfbd2bb3d30566ecae361c4abf6dbf

There is a little known feature in app_controlplayback that will cause the
specified offset to be used relative to the end of a file if a ':end' is
detected within the filename.

This feature is pretty bad, but okay.

However, a bug exists in this code where a ':' detected in the filename
will cause the end pointer to be non-NULL, even if the full ':end' isn't
specified. This causes us to treat an unspecified offset (0) as being
"start playing from the end of the file", resulting in no file playback
occurring.

This patch fixes this bug by resetting the end pointer if ':end' is not
found in the filename.

Change-Id: Ib4c7b1b45283e4effd622a970055c51146892f35

This patch allows applications/APIs that access media through the core file
APIs to play media in the media cache. Prior to determining if a 'filename'
exists, the filename is passed to the media cache's retrieve API call. If
that call succeeds, the local file specified passed back by the API is
opened for streaming. When used in this fashion, the 'filename' is actually
a URI that the media cache process and understand.

ASTERISK-25654 #close

Change-Id: I73b6e2e90c3e91b8500581c45cdf9c0dc785f5f0

This patch adds unit tests for res_http_media cache, that covers nominal
creation and retrieval - and through them as well, staleness and deletion
checks. In addition, this patch adds tests that covers the interaction of
various HTTP headers, including Expires, Etag, and Cache-Control.

ASTERISK-25654

Change-Id: I2db101e307c863857fe416d6f5bf4cace9ac7cf5

This patch adds a bucket backend for the core media cache that interfaces to a
remote HTTP server. When a media item is requested in the cache, the cache will
query its bucket backends to see if they can provide the media item. If that
media item has a scheme of HTTP or HTTPS, this backend will be invoked.

The backend provides callbacks for the following:
 * create - this will always retrieve the URI specified by the provided
            bucket_file, and store it in the file specified by the object.
 * retrieve - this will pull the URI specified and store it in a temporary
              file. It is then up to the media cache to move/rename this file
              if desired.
 * delete - destroys the file associated with the bucket_file.
 * stale - if the bucket_file has expired, based on received HTTP headers from
           the remote server, or if the ETag on the server no longer matches
           the ETag stored on the bucket_file, the resource is determined to be
           stale.

Note that the backend respects the ETag, Expires, and Cache-Control headers
provided by the HTTP server it is querying.

ASTERISK-25654

Change-Id: Ie201c2b34cafc0c90a7ee18d7c8359afaccc5250

This patch does the following:

First, it addresses file extension handling in the media cache. The media core
in Asterisk is a bit interesting in that it wants:
 * A file to have an extension on it. That extension is used to associate the
   file with a defined format module.
 * The filename passed to the core to not have an extension on it. This allows
   the core to match the available file formats with the format a channel
   is capable of handling.

Unfortunately, this makes the current implementation a bit lacking in the media
cache. By default, we do not store the extension of a retrieved URI on the
local file that is created. As a result, the media core does not know what
format the file is, and the file is ignored. Modifying the file outside of the
media core is bad, as we would not be able to update the internal
ast_bucket_file's path.

At the same time, we do not want to pass the extension out in the file_path
parameter in ast_media_cache_retrieve. This parameter is intended to be fed
into the media core; if we passed the extension, all callers would have to
strip it off.

Thus, this patch does the following:
* If there is an extension specified in the URL, we append it to the local
  file name (if a preferred file name isn't specified), and we store that
  in the local file path.
* The extension, however, is stripped off of the file_path parameter passed
  back out of ast_media_cache_retrieve.

Second, this patch causes stale items to be completely removed from the system.
Prior to this patch, sound files could be orphaned due to the bucket
referencing the file being deleted, but the file itself not being removed. This
is now addressed by explicitly calling ast_bucket_file_delete on the
bucket_file when it is deemed to be stale. Note that this only happen when we
know we will attempt to retrieve the resource again.

Finally, this patch changes the AO2 container holding media items to just use
a regular mutex. The usage for this container already assumed it was a plain
mutex, and - given that retrieval of an item can cause it to be replaced in
the container - a mutex makes more sense than a read/write lock.

Change-Id: I51667fff86ae8d2e4a663555dfa85b11e935fe0f

This patch adds a write option to the CURL dialplan function, allowing it to
CURL files and store them locally. The value 'written' to the CURL URL
specifies the location on disk to store the file. As an example:

same => n,Set(CURL(http://1.1.1.1/foo.wav)=/tmp/foo.wav)

Would retrieve the file foo.wav from the remote server and store it in the
/tmp directory.

Due to the potentially dangerous nature of this function call, APIs are
forbidden from using the write functionality unless live_dangerously is set
to True in asterisk.conf.

ASTERISK-25652 #close

Change-Id: I44f4ad823d7d20f04ceaad3698c5c7f653c41b0d

check_via() already skips leading blanks where the sent-by address (with the
optional port) should be placed.

Since RFC 3261 allows for blanks between the port ant the Via parameters:
> https://tools.ietf.org/html/rfc3261#section-20.42
(actually it allows a lot of blanks more ;-)). I just switched from
ast_skip_blanks() to ast_strip() on the local copy of the string.

ASTERISK-21301 #close

Change-Id: Ie5b8fe5a07067b7c0dc9bcdd1707e99b23b02b06

The encryption code for AES_ENCRYPT evaluates the length of the data to
be encoded in base64 using strlen. The data is binary, thus the length
of it can be underestimated at the first NULL character.
Reuse the write pointer offset to evaluate it, instead.

ASTERISK-25857 #close

Change-Id: If686b5d570473eb926693c73461177b35b13b186

Fix the reference leak introduced in the following commit:

c534bd58

ASTERISK-25849

Change-Id: I5cfefd5ee6c1c3a1715c050330aaa10e4d2a5e85

During a transfer involving direct media a race occurs between when the
transferer channel is swapped out, initiating rtp changes/updates, and the
subsequent reinvites.

When Alice, after speaking with Charlie (Bob is on hold), connects Bob and
Charlie invites are sent to each in order to establish the call between them.
Bob is taken off hold and Charlie is told to have his media flow through
Asterisk. However, if before those invites go out the bridge updates Bob's
and/or Charlie's rtp information with direct media data (i.e. address, port)
then the invite(s) will contain the remote data in the SDP instead of the
Asterisk data.

The race occurs in the native bridge glue code when updating the peer. The
direct_media_address can get set twice before sending out the first invite
during call connection. This can happen because the checking/setting of the
direct_media_address happened in one thread while the sending of the invite(s)
happened in another thread.

This fix removes the race condition by moving the checking/setting of the
direct_media_address to be in the same thread as the sending of the invites(s).
This serializes the checking/setting and sending so they can no longer happen
out of order.

ASTERISK-25849 #close

Change-Id: Idfea590175e74f401929a601dba0c91ca1a7f873

The "Refer-to" header of an incoming REFER request is parsed by
pjsip_parse_uri().  That function requires the URI parameter to be NULL
terminated.  Unfortunately, the previous code added the NULL terminator by
overwriting memory that may not be safe.  The overwritten memory results
could be benign, memory corruption, or a segmentation fault.  Now the URI
is NULL terminated safely by copying the URI to a new chunk of memory with
the correct size to be NULL terminated.

ASTERISK-25814 #close

Change-Id: I32565496684a5a49c3278fce06474b8c94b37342

This work-in-progress is the first step to being able to reliably
build Asterisk containers from the Asterisk source. I'm submitting
this based on feedback gained at AstriDevCon 2015.

Information about how to use this is provided in contrib/docker/README.md
and will result in a local Asterisk container being built right from
your source. I believe this can eventually be automated via
hub.docker.com.

Change-Id: Ifa070706d40e56755797097b6ed72c1e243bd0d1

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023 #close

Change-Id: I96d429c57a48861fd8bde63dd93db4e92dc3adb6

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023

Change-Id: I2e40de89efc8ae6e8850771d089ca44bc604b508

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

* Made always run check_pendings() under the scheduler thread so scheduler
ids can be checked safely.

ASTERISK-25023

Change-Id: Ia834d6edd5bdb47c163e4ecf884428a4a8b17d52

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023

Change-Id: If595e4456cd059d7171880c7f354e844c21b5f5f

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023

Change-Id: I6d65269151ba95e0d8fe4e9e611881cde2ab4900

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023

Change-Id: I9c11b9d597468f63916c99e1dabff9f4a46f84c1

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

* Fix clearing autokillid in __sip_autodestruct() even though we could
reschedule.

ASTERISK-25023

Change-Id: I450580dbf26e2e3952ee6628c735b001565c368f

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

* Fix retrans_pkt() to call check_pendings() with both the owner channel
and the private objects locked as required.

* Refactor dialog retransmission packet list to safely remove packet
nodes.  The list nodes are now ao2 objects.  The list has a ref and the
scheduled entry has a ref.

ASTERISK-25023

Change-Id: I50926d81be53f4cd3d572a3292cd25f563f59641

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023

Change-Id: I98a694fd42bc81436c83aa92de03226e6e4e3f48

This patch is part of a series to resolve deadlocks in chan_sip.c.

* Make dialog_unlink_all() unschedule all items at once in the sched
thread.

ASTERISK-25023

Change-Id: I7743072fb228836e8228b72f6dc46c8cc50b3fb4

This patch is part of a series to resolve deadlocks in chan_sip.c.

The reordering of chan_sip's shutdown is to handle any immediate events
that get put onto the scheduler so resources aren't leaked.  The typical
immediate events at this time are going to be concerned with stopping
other scheduled events.

ASTERISK-25023

Change-Id: I3f6540717634f6f2e84d8531a054976f2bbb9d20

This patch is part of a series to resolve deadlocks in chan_sip.c.

Delaying destruction of the chan_sip sip_pvt structures caused the
/channels/chan_sip/test_sip_rtpqos unit test to crash.  That test
registers a special test ast_rtp_engine with the rtp engine module.  When
the unit test completes it cleans up by unregistering the test
ast_rtp_engine and exits.  Since the delayed destruction of the sip_pvt
happens after the unit test returns, the destructor tries to call the rtp
engine destroy callback of the test ast_rtp_engine auto variable which no
longer exists on the stack.

* Change the test ast_rtp_engine auto variable to a static variable.  Now
the variable can still exist after the unit test exits so the delayed
sip_pvt destruction can complete successfully.

ASTERISK-25023

Change-Id: I61e34a12d425189ef7e96fc69ae14993f82f3f13

This patch is part of a series to resolve deadlocks in chan_sip.c.

* Updated sched unit test to check new behavior.

ASTERISK-25023

Change-Id: Ib69437327b3cda5e14c4238d9ff91b2531b34ef3