This patch adds a bucket backend for the core media cache that interfaces to a
remote HTTP server. When a media item is requested in the cache, the cache will
query its bucket backends to see if they can provide the media item. If that
media item has a scheme of HTTP or HTTPS, this backend will be invoked.

The backend provides callbacks for the following:
 * create - this will always retrieve the URI specified by the provided
            bucket_file, and store it in the file specified by the object.
 * retrieve - this will pull the URI specified and store it in a temporary
              file. It is then up to the media cache to move/rename this file
              if desired.
 * delete - destroys the file associated with the bucket_file.
 * stale - if the bucket_file has expired, based on received HTTP headers from
           the remote server, or if the ETag on the server no longer matches
           the ETag stored on the bucket_file, the resource is determined to be
           stale.

Note that the backend respects the ETag, Expires, and Cache-Control headers
provided by the HTTP server it is querying.

ASTERISK-25654

Change-Id: Ie201c2b34cafc0c90a7ee18d7c8359afaccc5250

This patch does the following:

First, it addresses file extension handling in the media cache. The media core
in Asterisk is a bit interesting in that it wants:
 * A file to have an extension on it. That extension is used to associate the
   file with a defined format module.
 * The filename passed to the core to not have an extension on it. This allows
   the core to match the available file formats with the format a channel
   is capable of handling.

Unfortunately, this makes the current implementation a bit lacking in the media
cache. By default, we do not store the extension of a retrieved URI on the
local file that is created. As a result, the media core does not know what
format the file is, and the file is ignored. Modifying the file outside of the
media core is bad, as we would not be able to update the internal
ast_bucket_file's path.

At the same time, we do not want to pass the extension out in the file_path
parameter in ast_media_cache_retrieve. This parameter is intended to be fed
into the media core; if we passed the extension, all callers would have to
strip it off.

Thus, this patch does the following:
* If there is an extension specified in the URL, we append it to the local
  file name (if a preferred file name isn't specified), and we store that
  in the local file path.
* The extension, however, is stripped off of the file_path parameter passed
  back out of ast_media_cache_retrieve.

Second, this patch causes stale items to be completely removed from the system.
Prior to this patch, sound files could be orphaned due to the bucket
referencing the file being deleted, but the file itself not being removed. This
is now addressed by explicitly calling ast_bucket_file_delete on the
bucket_file when it is deemed to be stale. Note that this only happen when we
know we will attempt to retrieve the resource again.

Finally, this patch changes the AO2 container holding media items to just use
a regular mutex. The usage for this container already assumed it was a plain
mutex, and - given that retrieval of an item can cause it to be replaced in
the container - a mutex makes more sense than a read/write lock.

Change-Id: I51667fff86ae8d2e4a663555dfa85b11e935fe0f

This patch adds a write option to the CURL dialplan function, allowing it to
CURL files and store them locally. The value 'written' to the CURL URL
specifies the location on disk to store the file. As an example:

same => n,Set(CURL(http://1.1.1.1/foo.wav)=/tmp/foo.wav)

Would retrieve the file foo.wav from the remote server and store it in the
/tmp directory.

Due to the potentially dangerous nature of this function call, APIs are
forbidden from using the write functionality unless live_dangerously is set
to True in asterisk.conf.

ASTERISK-25652 #close

Change-Id: I44f4ad823d7d20f04ceaad3698c5c7f653c41b0d

The encryption code for AES_ENCRYPT evaluates the length of the data to
be encoded in base64 using strlen. The data is binary, thus the length
of it can be underestimated at the first NULL character.
Reuse the write pointer offset to evaluate it, instead.

ASTERISK-25857 #close

Change-Id: If686b5d570473eb926693c73461177b35b13b186

Fix the reference leak introduced in the following commit:

c534bd58

ASTERISK-25849

Change-Id: I5cfefd5ee6c1c3a1715c050330aaa10e4d2a5e85

During a transfer involving direct media a race occurs between when the
transferer channel is swapped out, initiating rtp changes/updates, and the
subsequent reinvites.

When Alice, after speaking with Charlie (Bob is on hold), connects Bob and
Charlie invites are sent to each in order to establish the call between them.
Bob is taken off hold and Charlie is told to have his media flow through
Asterisk. However, if before those invites go out the bridge updates Bob's
and/or Charlie's rtp information with direct media data (i.e. address, port)
then the invite(s) will contain the remote data in the SDP instead of the
Asterisk data.

The race occurs in the native bridge glue code when updating the peer. The
direct_media_address can get set twice before sending out the first invite
during call connection. This can happen because the checking/setting of the
direct_media_address happened in one thread while the sending of the invite(s)
happened in another thread.

This fix removes the race condition by moving the checking/setting of the
direct_media_address to be in the same thread as the sending of the invites(s).
This serializes the checking/setting and sending so they can no longer happen
out of order.

ASTERISK-25849 #close

Change-Id: Idfea590175e74f401929a601dba0c91ca1a7f873

The "Refer-to" header of an incoming REFER request is parsed by
pjsip_parse_uri().  That function requires the URI parameter to be NULL
terminated.  Unfortunately, the previous code added the NULL terminator by
overwriting memory that may not be safe.  The overwritten memory results
could be benign, memory corruption, or a segmentation fault.  Now the URI
is NULL terminated safely by copying the URI to a new chunk of memory with
the correct size to be NULL terminated.

ASTERISK-25814 #close

Change-Id: I32565496684a5a49c3278fce06474b8c94b37342

This work-in-progress is the first step to being able to reliably
build Asterisk containers from the Asterisk source. I'm submitting
this based on feedback gained at AstriDevCon 2015.

Information about how to use this is provided in contrib/docker/README.md
and will result in a local Asterisk container being built right from
your source. I believe this can eventually be automated via
hub.docker.com.

Change-Id: Ifa070706d40e56755797097b6ed72c1e243bd0d1

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023 #close

Change-Id: I96d429c57a48861fd8bde63dd93db4e92dc3adb6

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023

Change-Id: I2e40de89efc8ae6e8850771d089ca44bc604b508

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

* Made always run check_pendings() under the scheduler thread so scheduler
ids can be checked safely.

ASTERISK-25023

Change-Id: Ia834d6edd5bdb47c163e4ecf884428a4a8b17d52

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023

Change-Id: If595e4456cd059d7171880c7f354e844c21b5f5f

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023

Change-Id: I6d65269151ba95e0d8fe4e9e611881cde2ab4900

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023

Change-Id: I9c11b9d597468f63916c99e1dabff9f4a46f84c1

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

* Fix clearing autokillid in __sip_autodestruct() even though we could
reschedule.

ASTERISK-25023

Change-Id: I450580dbf26e2e3952ee6628c735b001565c368f

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

* Fix retrans_pkt() to call check_pendings() with both the owner channel
and the private objects locked as required.

* Refactor dialog retransmission packet list to safely remove packet
nodes.  The list nodes are now ao2 objects.  The list has a ref and the
scheduled entry has a ref.

ASTERISK-25023

Change-Id: I50926d81be53f4cd3d572a3292cd25f563f59641

This patch is part of a series to resolve deadlocks in chan_sip.c.

Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event.  If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen.  The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event.  Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.

ASTERISK-25023

Change-Id: I98a694fd42bc81436c83aa92de03226e6e4e3f48

This patch is part of a series to resolve deadlocks in chan_sip.c.

* Make dialog_unlink_all() unschedule all items at once in the sched
thread.

ASTERISK-25023

Change-Id: I7743072fb228836e8228b72f6dc46c8cc50b3fb4

This patch is part of a series to resolve deadlocks in chan_sip.c.

The reordering of chan_sip's shutdown is to handle any immediate events
that get put onto the scheduler so resources aren't leaked.  The typical
immediate events at this time are going to be concerned with stopping
other scheduled events.

ASTERISK-25023

Change-Id: I3f6540717634f6f2e84d8531a054976f2bbb9d20

This patch is part of a series to resolve deadlocks in chan_sip.c.

Delaying destruction of the chan_sip sip_pvt structures caused the
/channels/chan_sip/test_sip_rtpqos unit test to crash.  That test
registers a special test ast_rtp_engine with the rtp engine module.  When
the unit test completes it cleans up by unregistering the test
ast_rtp_engine and exits.  Since the delayed destruction of the sip_pvt
happens after the unit test returns, the destructor tries to call the rtp
engine destroy callback of the test ast_rtp_engine auto variable which no
longer exists on the stack.

* Change the test ast_rtp_engine auto variable to a static variable.  Now
the variable can still exist after the unit test exits so the delayed
sip_pvt destruction can complete successfully.

ASTERISK-25023

Change-Id: I61e34a12d425189ef7e96fc69ae14993f82f3f13

This patch is part of a series to resolve deadlocks in chan_sip.c.

* Updated sched unit test to check new behavior.

ASTERISK-25023

Change-Id: Ib69437327b3cda5e14c4238d9ff91b2531b34ef3

This prevents pbx_core from hanging up the channel if the app isn't
registered.

ASTERISK-25846 #close

Change-Id: I63216a61f30706d5362bc0906b50b6f0544aebce

Remove destructor calling destroy_it calling really_destroy_it
for no benefit.  Just make the destructor the really_destroy_it
function.

Change-Id: Idea0d47b27dd74f2488db75bcc7f353d8fdc614a

Change-Id: I90f04208a089f95488a2460185a8dbc3f6acca12

Older versions of PJSIP do not have the proto field on the TLS transport
setting structure. This change adds a configure check so even if it is
not present we will still be able to build.

Change-Id: Ibf3f47befb91ed1b8194bf63888baa6fee05aba9

I can't ever recall actually needing the intermediate files or the checking
that a double compile produces.  What I CAN remember is every DONT_OPTIMIZE
build needing 3 invocations of gcc instead of 1 just to do the checks and
produce those intermediate files.

Having said that, Richard pointed out that the reason for the double compile
was that there were cases in the past where a submitted patch failed to compile
because the submitter never tried it with the optimizations turned on.

To get the best of both worlds, COMPILE_DOUBLE has been split into its own
option.  If DONT_OPTIMIZE is turned on, COMPILE_DOUBLE will also be selected
BUT you can then turn it off if all you need are the debugging symbols.  This
way you have to make an informed decision about disabling COMPILE_DOUBLE.

To allow COMPILE_DOUBLE to be both auto-selected and turned off, a new feature
was added to menuselect.  The <use> element can now contain an "autoselect"
attribute which will turn the used member on but not create a hard dependency.
The cflags.xml implementation for COMPILE_DOUBLE looks like this...

<member name="DONT_OPTIMIZE" displayname="Disable Optimizations ...">
	<use autoselect="yes">COMPILE_DOUBLE</use>
	<support_level>core</support_level>
</member>
<member name="COMPILE_DOUBLE" displayname="Pre-compile with ...>
	<depend>DONT_OPTIMIZE</depend>
	<support_level>core</support_level>
</member>

When DONT_OPTIMIZE is turned on, COMPILE_DOUBLE is turned on because
of the use.
When DONT_OPTIMIZE is turned off, COMPILE_DOUBLE is turned off because
of the depend.
When COMPILE_DOUBLE is turned on, DONT_OPTIMIZE is turned on because
of the depend.
When COMPILE_DOUBLE is turned off, DONT_OPTIMIZE is left as is because
it only uses COMPILE_DOUBLE, it doesn't depend on it.

I also made a few tweaks to the ncurses implementation to move things
left a bit to allow longer descriptions.

Change-Id: Id49ca930ac4b5ec4fc2d8141979ad888da7b1611

The pjproject Makefile now uses the Asterisk optimization flags which
are determined by the setting of the DONT_OPTMIZE menuselect flag.
The Makefile was also restructured so a change to the top level
menuselect.makeopts will result in a rebuild of pjproject.

Also, "--disable-resample" was removed from the pjproject configure
options.  Without resample, pjsua (which is used by the testsuite)
can't make audio calls.  When it can't, it segfaults.

Change-Id: I24b0a4d0872acef00ed89b3c527a713ee4c2ccd4

Channel masquerading had a conflict with autochannel locking.

When locking autochannel->channel, the channel is fetched from the
autochannel and then locked. During the fetch, the autochannel -- which
has no locks itself -- can be modified by someone who owns the channel
lock. That means that the value of autochan->channel cannot be trusted
until you hold the lock.

In practice, this caused problems with Local channels getting
masqueraded away while the ChanSpy attempted to get info from that
channel. The old channel which was about to get removed got locked, but
the new (replaced) channel got unlocked (no-op). Because the replaced
channel was now locked (and would never get unlocked), it couldn't get
removed from the channel list in a timely manner, and would now cause
deadlocks when iterating over the channel list.

This change checks the autochannel after locking the channel for changes
to the autochannel. If the channel had been changed, the lock is
reobtained on the new channel.

In theory it seems possible that after this fix, the lock attempt on the
old (wrong) channel can be on an already destroyed lock, maybe causing
a crash. But that hasn't been observed in the wild and is harder induce
than the current deadlock.

Thanks go to Filip Frank for suggesting a fix similar to this and
especially to IRC user hexanol for pointing out why this deadlock was
possible and testing this fix. And to Richard for catching my rookie
while loop mistake ;)

ASTERISK-25321 #close

Change-Id: I293ae0014e531cd0e675c3f02d1d118a98683def