Two changes of note in this patch:

* Use ast_file_read_dir instead of opendir/readdir/closedir

* If the files list should be sorted, do that at the end rather than as
  we go which improves performance for large lists

Change-Id: Ic7e9c913c0f85754c99c74c9cf6dd3514b1b941f

The SCOPE_ENTER and SCOPE_EXIT* macros now print debug messages
at the same level as the scope level.  This allows the same
messages to be printed to the debug log when AST_DEVMODE
isn't enabled.

Also added a few variants of the SCOPE_EXIT macros that will
also call ast_log instead of ast_debug to make it easier to
use scope tracing and still print error messages.

Change-Id: I7fe55f7ec28069919a0fc0b11a82235ce904cc21

 * Added ast_stream_to_stra and ast_stream_topology_to_stra() macros
   which are shortcuts for
      ast_str_tmp(256, ast_stream_to_str(stream, &STR_TMP))

 * Added the stream position to the string representation of the
   stream.

 * Fixed some formatting in ast_stream_to_str().

Change-Id: Idaf4cb0affa46d4dce58a73a111f35435331cc4b

The ToHost parameter was not cleared when a peer's host value was
changed to dynamic. This causes invites to be sent to the original host.

ASTERISK-29011 #close

Change-Id: I9678d512741f71baca8f131a65b7523020b07d5c

Allow passing a topology from the called channel back to the
calling channel.

 * Added a new function ast_queue_answer() that accepts a stream
   topology and queues an ANSWER CONTROL frame with it as the
   data.  This allows the called channel to indicate its resolved
   topology.

 * Added a new virtual function to the channel tech structure
   answer_with_stream_topology() that allows the calling channel
   to receive the called channel's topology.  Added
   ast_raw_answer_with_stream_topology() that invokes that virtual
   function.

 * Modified app_dial.c and features.c to grab the topology from the
   ANSWER frame queued by the answering channel and send it to
   the calling channel with ast_raw_answer_with_stream_topology().

 * Modified frame.c to automatically cleanup the reference
   to the topology on ANSWER frames.

Added a few debugging messages to stream.c.

Change-Id: I0115d2ed68d6bae0f87e85abcf16c771bdaf992c

Adds sed before awk to produce reasonable ASTERISKVERSIONNUM
on certified versions of Asterisk eg. 16.8-cert3 is 160803
instead of the previous 00800.

ASTERISK-29021 #close

Change-Id: Icf241df0ff6db09011b8c936a317a84b0b634e16

The MoH class internal file vector is potentially being manipulated by
multiple threads at the same time without sufficient locking. Switch to
a reference counted list and operate on copies where necessary.

ASTERISK-28927 #close

Change-Id: I479c5dcf88db670956e8cac177b5826c986b0217

When reading in a codec preference configuration option
the value would be set on the respective option before
applying any default adjustments, resulting in the
configuration not being as expected.

This was exposed by the REST API push configuration as
it used the configuration returned by Asterisk to then do
a modification. In the case of codec preferences one of
the options had a transcode value of "unspecified" when the
defaults should have ensured it would be "allow" instead.

This also renames the options in other places that were
missed.

Change-Id: I4ad42e74fdf181be2e17bc75901c62591d403964

The assumed behavior of realloc() - that it was effectively a free() if
its second argument was 0 - is Linux specific behavior and is not
guaranteed by either POSIX or the C specification.

Instead, if we want to resize a vector to 0, do it explicitly.

Change-Id: Ife31d4b510ebab41cb5477fdc7ea4e3138ca8b4f

PJSIP, UDP transport with external_media_address and session timers
enabled. Connected to SIP server that is not in local net. Asterisk
initiated the connection and is refreshing the session after 150s
(timeout 300s). The 2nd refresh-INVITE triggered by the pjsip timer has
a malformed IP address in its SDP (garbage string). This only happens
when the SDP is modified by the nat-code to replace the local IP address
with the configured external_media_address.
Analysis: the code to modify the SDP (in
res_pjsip_session.c:session_outgoing_nat_hook() and also (redundantly?)
in res_pjsip_sdp_rtp.c:change_outgoing_sdp_stream_media_address()) uses
the tdata->pool to allocate the replacement string. But the same
pjmedia_sdp_stream that was modified for the 1st refresh-INVITE is also
used for the 2nd refresh-INVITE (because it is stored in pjmedia's
pjmedia_sdp_neg structure). The problem is, that at that moment, the
tdata->pool that holds the stringified external_media_address from the
1. refresh-INVITE has long been reused for something else.
Fix by Sauw Ming of pjproject (see
https://github.com/pjsip/pjproject/pull/2476): the local, potentially
modified pjmedia_sdp_stream is cloned in
pjproject/source/pjsip/src/pjmedia/sip_neg.c:process_answer() and the
clone is stored, thereby detaching from the tdata->pool (which is only
released *after* process_answer())

ASTERISK-28973
Reported-by: Michael Neuhauser

Change-Id: I272ac22436076596e06aa51b9fa23fd1c7734a0e

With the addition of STIR/SHAKEN, the function ast_base64decode_string
was added for convenience since there is a lot of converting done during
the STIR/SHAKEN process. This function returned the decoded string for
you, but did not NULL terminate it, causing some issues (specifically
with MALLOC_DEBUG). Now, the returned string is NULL terminated, and the
documentation has been updated to reflect this.

Change-Id: Icdd7d05b323b0c47ff6ed43492937a03641bdcf5

This change renames the codec preference endpoint options.
incoming_offer_codec_prefs becomes codec_prefs_incoming_offer
to keep the options together when showing an endpoint.

Change-Id: I6202965b4723777f22a83afcbbafcdafb1d11c8d

Fixed a memory allocation that was not passing in the correct size for
the struct in curl.c.

Change-Id: I5fb92fbbe84b075fa6aefa2423786df80e114c3a
(cherry picked from commit deaa3742)

When a bundled stream is removed, its bundle_group is reset to -1.
If that stream is later reused, the bundle parameters on session
media need to be reset correctly it could mistakenly be rebundled
with a stream that was removed and never reused.  Since the removed
stream has no rtp instance, a crash will result.

Change-Id: Ie2b792220f9291587ab5f9fd123145559dba96d7

Statically configured contacts on an AOR don't have an expiration
time so when adding them to the resulting 200 OK if an endpoint
registers ensure they are marked as such.

ASTERISK-28995

Change-Id: I9f0e45eb2ccdedc9a0df5358634a19ccab0ad596

There are various places in Asterisk - specifically in regards to
database integration - where having some kind of UTF-8 validation would
be beneficial. This patch adds:

* Functions to validate that a given string contains only valid UTF-8
  sequences.

* A function to copy a string (similar to ast_copy_string) stopping when
  an invalid UTF-8 sequence is encountered.

* A UTF-8 validator that allows for progressive validation.

All of this is based on the excellent UTF-8 decoder by Björn Höhrmann.
More information is available here:

    https://bjoern.hoehrmann.de/utf-8/decoder/dfa/

The API was written in such a way that should allow us to replace the
implementation later should we determine that we need something more
comprehensive.

Change-Id: I3555d787a79e7c780a7800cd26e0b5056368abf9

Currently, if the bridge has created by the ARI, the video_mode
parameter was
not shown in the BridgeCreated event correctly.

Fixed it and added video_mode shown in the 'bridge show <bridge id>'
cli.

ASTERISK-28987

Change-Id: I8c205126724e34c2bdab9380f523eb62478e4295

Allows a vector to be sorted in-place, rather than only during
insertion.

Change-Id: I22cba9ddf556a7e44dacc53c4431bd81dd2fa780

Change-Id: I7d951e75ad2d472fa096647dfb55670b11105e23

When dealing with a lot of video streams on WebRTC
the resulting SDPs can grow to be quite large. This
effectively doubles the maximum size to allow more
streams to exist.

The res_http_websocket module has also been changed
to use a buffer on the session for reading in packets
to ensure that the stack space usage is not excessive.

Change-Id: I31d4351d70c8e2c11564807a7528b984f3fbdd01

If an ACL is misconfigured in the realtime database (for instance, the
"rule" is blank) and Asterisk attempts to read the ACL, Asterisk will
crash.

ASTERISK-28978 #close

Change-Id: Ic1536c4df856231bfd2da00128f7822224d77610

I noticed this while looking at another issue and brought
it up with Teluu. It was possible for an uninitialized timer
to be cancelled, resulting in the invalid timer id of 0
being placed into the timer heap causing issues.

This change is a backport from the pjproject repository
preventing this from happening.

Change-Id: I1ba318b1f153a6dd7458846396e2867282b428e7

Change-Id: Ib2c42fc2d46563e2fbadbd5513cb029b4042791e

We read beyond the end of the buffer when copying the string out of the
buffer when we used ast_copy_string() because the original string was
not null terminated. Instead switch to ast_strndup() which does not
exhibit the same behavior.

ASTERISK-28975 #close

Change-Id: Ib4a75cffeb1eb8cf01136ef30306bd623e531a2a

Added a new configuration option for PJSIP endpoints - stir_shaken. If
set to yes, then STIR/SHAKEN support will be added to inbound and
outbound INVITEs. The default is no. Alembic has been updated to include
this option.

Previously the dialplan function was not trimming the whitespace from
the parameters it recieved. Now it does.

Also added a conditional that, when TEST_FRAMEWORK is enabled, the
timestamp in the identity header will be overlooked. This is just for
testing, since the testsuite will rely on a SIPp scenario with a preset
identity header to trigger the MISMATCH result.

Change-Id: I43d67f1489b8c1c5729ed3ca8d71e35ddf438df1

session_on_rx_response wasn't checking for a NULL dialog before
attempting to get the invite session from it.

Change-Id: Id13534375966cc2eb7f2b55717c9813c63c10065

If your queues.conf had _no_ [general] section, they would default to
'yes'. Now, they always default to 'no'.

(Actually, commit ed615afb already
partially fixed it for shared_lastcall.)

ASTERISK-28951

Change-Id: Ic39d8a0202906bc454194368bbfbae62990fe5f6

Prior to making any modifications to the pjsip infrastructure
for ACN, I've added the tracing functions to the existing code.
This should make the final commit easier to review, but we can also
now run a "before and after" trace.

No functional changes were made with this commit.

Change-Id: Ia83a1a2687ccb96f2bc8a2a3928a5214c4be775c

This commit adds the endpoint options required to control
Advanced Codec Negotiation.

incoming_offer_codec_prefs
outgoing_offer_codec_prefs
incoming_answer_codec_prefs
outgoing_answer_codec_prefs

The documentation may need tweaking and some additional edits
added, especially for the "answer" prefs.  That'll be handled
when things finalize.

This commit is safe to merge as it doens't alter any existing
functionality nor does it alter the previous codec negotiation
work which may now be obsolete.

Change-Id: I920ba925d7dd36430dfd2ebd9d82d23f123d0e11

Currently when the pjsip making an outgoing request, it keep adding the
rport parameter in a request message as a default.

This causes unexpected rport handle at the other end.

Added option for disable this behaviour in the pjsip.conf.

This is a system option, but working as a gloabl option.

ASTERISK-28959

Change-Id: I9596675e52a742774738b5aad5d1fec32f477abc

 * ast_frame_subclass2str() and ast_frame_type2str() now return
   a pointer to the buffer that was passed in instead of void.
   This makes it easier to use these functions inline in
   printf-style debugging statements.

 * Added many missing control frame entries in
   ast_frame_subclass2str.

Change-Id: Ifd0d6578e758cd644c96d17a5383ff2128c572fc

Tracing through synchronous tasks was a little troublesome because
the new thread's stack counter reset to 0.  This change allows
a synchronous task to set its trace level to be the same as the
thread that pushed the task.  For now, the task's level has to be
passed in the task's data structure but a future enhancement to the
taskprocessor subsystem could automatically set the trace level
of the servant to be that of the caller.

This doesn't really make sense for async tasks because you never
know when they're going to run anyway.

Change-Id: Ib8049c0b815063a45d8c7b0cb4e30b7b87b1d825

Do not return error if the client received ping frame
while looking for a string and just wait for another frame.

ASTERISK-28958 #close

Change-Id: I4d06b4827bd71e56cbaafc011ffdcef9f0332922

When using the PSJIP_MEDIA_OFFER dialplan function it was not
overriding an endpoint's configured codecs on refresh unless
they had a shared codec between the two.

This patch makes it so whatever is set using PJSIP_MEDIA_OFFER
is used when creating the SDP for a refresh no matter what.

ASTERISK-28878 #close

Change-Id: I0f7dc86fd0fb607c308e6f98ede303c54d1eacb6

This patch allows a user of AMI to now specify the type of message
content contained within by setting the 'Content-Type' parameter.

Note, the AMI version has been bumped for this change.

ASTERISK-28945 #close

Change-Id: Ibb5315702532c6b954e1498beddc8855fabdf4bb

The Streams API becomes the home for the core ACN capabilities.
These include...

 * Parsing and formatting of codec negotation preferences.
 * Resolving pending streams and topologies with those configured
   using configured preferences.
 * Utility functions for creating string representations of
   streams, topologies, and negotiation preferences.

For codec negotiation preferences:
 * Added ast_stream_codec_prefs_parse() which takes a string
   representation of codec negotiation preferences, which
   may come from a pjsip endpoint for example, and populates
   a ast_stream_codec_negotiation_prefs structure.
 * Added ast_stream_codec_prefs_to_str() which does the reverse.
 * Added many functions to parse individual parameter name
   and value strings to their respectrive enum values, and the
   reverse.

For streams:
 * Added ast_stream_create_resolved() which takes a "live" stream
   and resolves it with a configured stream and the negotiation
   preferences to create a new stream.
 * Added ast_stream_to_str() which create a string representation
   of a stream suitable for debug or display purposes.

For topology:
 * Added ast_stream_topology_create_resolved() which takes a "live"
   topology and resolves it, stream by stream, with a configured
   topology stream and the negotiation preferences to create a new
   topology.
 * Added ast_stream_topology_to_str() which create a string
   representation of a topology suitable for debug or display
   purposes.
 * Renamed ast_format_caps_from_topology() to
   ast_stream_topology_get_formats() to be more consistent with
   the existing ast_stream_get_formats().

Additional changes:
 * A new function ast_format_cap_append_names() appends the results
   to the ast_str buffer instead of replacing buffer contents.

Change-Id: I2df77dedd0c72c52deb6e329effe057a8e06cd56

Created new SCOPE_ functions that don't depend on RAII_VAR.  Besides
generating less code, the use of the explicit SCOPE_EXIT macros
capture the line number where the scope exited.  The RAII_VAR
versions can't do that.

 * SCOPE_ENTER(level, ...): Like SCOPE_TRACE but doesn't use
   RAII_VAR and therefore needs needs one of...

 * SCOPE_EXIT(...): Decrements the trace stack counter and optionally
   prints a message.

 * SCOPE_EXIT_EXPR(__expr, ...): Decrements the trace stack counter,
   optionally prints a message, then executes the expression.
   SCOPE_EXIT_EXPR(break, "My while got broken\n");

 * SCOPE_EXIT_RTN(, ...): Decrements the trace stack counter,
   optionally prints a message, then returns without a value.
   SCOPE_EXIT_RTN("Bye\n");

 * SCOPE_EXIT_RTN_VALUE(__return_value, ...): Decrements the trace
   stack counter, optionally prints a message, then returns the value
   specified.
   SCOPE_EXIT_RTN_VALUE(rc, "Returning with RC: %d\n", rc);

Create an ast_str helper ast_str_tmp() that allocates a temporary
ast_str that can be passed to a function that needs it, then frees
it.  This makes using the above macros easier.  Example:

   SCOPE_ENTER(1, Format Caps 1: %s  Format Caps 2: %s\n",
       ast_str_tmp(32, ast_format_cap_get_names(cap1, &STR_TMP),
       ast_str_tmp(32, ast_format_cap_get_names(cap2, &STR_TMP));

The calls to ast_str_tmp create an ast_str of the specified initial
length which can be referenced as STR_TMP.  It then calls the
expression, which must return a char *, ast_strdupa's it, frees
STR_TMP, then returns the ast_strdupa'd string.  That string is
freed when the function returns.

Change-Id: I44059b20d55a889aa91440d2f8a590865998be51

The outbound proxy for an AOR was not being applied to
any statically configured Contacts. This resulted in the
OPTIONS requests being sent to the wrong target.

This change sets the outbound proxy on statically configured
contacts once the AOR configuration is done being
applied.

ASTERISK-28965

Change-Id: Ia60f3e93ea63f819c5a46bc8b54be2e588dfa9e0

Given a scenario where a module has a dependency on both
an external library and a module if the external library was
available and the module was not an infinite loop would
occur. This happened due to the code changing the dependecy
status to no failure on each dependency checking loop
iteration, resulting in the code thinking that it had
gone from no failure to failure each time triggering another
dependency check.

This change makes it so that the old dependency status is
preserved throughout the dependency checking allowing it to
determine that after the first iteration the dependency
status does not transition from no failure to failure.

ASTERISK-28930

Change-Id: Iea06d45d9fd6d8bfd068882a0bb7e23a53ec3e84