ASTERISK-28416 #close

Change-Id: I069420875ebdbcaada52d92599a5f7de3cb2cdf4

The recent 491 changes introduced a check to determine if the active
and pending topologies were equal and to suppress the re-invite if they
were. When a re-invite is sent for a COLP-only change, the pending
topology is NULL so that check doesn't happen and the re-invite is
correctly sent. Of course, sending the re-invite sets the pending
topology.  If a 491 is received, when we resend the re-invite, the
pending topology is set and since we didn't request a change to the
topology in the first place, pending and active topologies are equal so
the topologies-equal check causes the re-invite to be erroneously
suppressed.

This change checks if the topologies are equal before we run the media
state resolver (which recreates the pending topology) so that when we
do the final topologies-equal check we know if this was a topology
change request.  If it wasn't a change request, we don't suppress
the re-invite even though the topologies are equal.

ASTERISK-29014

Change-Id: Iffd7dd0500301156a566119ebde528d1a9573314

Added to:
 * bridges/bridge_softmix.c
 * channels/chan_pjsip.c
 * include/asterisk/res_pjsip_session.h
 * main/channel.c
 * res/res_pjsip_session.c

There NO functional changes in this commit.

Change-Id: I06af034d1ff3ea1feb56596fd7bd6d7939dfdcc3

When both Asterisk and a UA send re-invites at the same time, both
send 491 "Transaction in progress" responses to each other and back
off a specified amount of time before retrying. When Asterisk
prepares to send its re-invite, it sets up the session's pending
media state with the new topology it wants, then sends the
re-invite.  Unfortunately, when it received the re-invite from the
UA, it partially processed the media in the re-invite and reset
the pending media state before sending the 491 losing the state it
set in its own re-invite.

Asterisk also was not tracking re-invites received while an existing
re-invite was queued resulting in sending stale SDP with missing
or duplicated streams, or no re-invite at all because we erroneously
determined that a re-invite wasn't needed.

There was also an issue in bridge_softmix where we were using a stream
from the wrong topology to determine if a stream was added.  This also
caused us to erroneously determine that a re-invite wasn't needed.

Regardless of how the delayed re-invite was triggered, we need to
reconcile the topology that was active at the time the delayed
request was queued, the pending topology of the queued request,
and the topology currently active on the session.  To do this we
need a topology resolver AND we need to make stream named unique
so we can accurately tell what a stream has been added or removed
and if we can re-use a slot in the topology.

Summary of changes:

 * bridge_softmix:
   * We no longer reset the stream name to "removed" in
     remove_all_original_streams().  That was causing  multiple streams
     to have the same name and wrecked the checks for duplicate streams.

   * softmix_bridge_stream_sources_update() was checking the old_stream
     to see if it had the softmix prefix and not considering the stream
     as "new" if it did.  If the stream in that slot has something in it
     because another re-invite happened, then that slot in old might
     have a softmix stream but the same stream in new might actually
     be a new one.  Now we check the new_stream's name instead of
     the old_stream's.

 * stream:
   * Instead of using plain media type name ("audio", "video", etc) as
     the default stream name, we now append the stream position to it
     to make it unique.  We need to do this so we can distinguish multiple
     streams of the same type from each other.

   * When we set a stream's state to REMOVED, we no longer reset its
     name to "removed" or destroy its metadata.  Again, we need to
     do this so we can distinguish multiple streams of the same
     type from each other.

 * res_pjsip_session:
   * Added resolve_refresh_media_states() that takes in 3 media states
     and creates an up-to-date pending media state that includes the changes
     that might have happened while a delayed session refresh was in the
     delayed queue.

   * Added is_media_state_valid() that checks the consistency of
     a media state and returns a true/false value. A valid state has:
     * The same number of stream entries as media session entries.
         Some media session entries can be NULL however.
     * No duplicate streams.
     * A valid stream for each non-NULL media session.
     * A stream that matches each media session's stream_num
       and media type.

   * Updated handle_incoming_sdp() to set the stream name to include the
     stream position number in the name to make it unique.

   * Updated the ast_sip_session_delayed_request structure to include both
     the pending and active media states and updated the associated delay
     functions to process them.

   * Updated sip_session_refresh() to accept both the pending and active
     media states that were in effect when the request was originally queued
     and to pass them on should the request need to be delayed again.

   * Updated sip_session_refresh() to call resolve_refresh_media_states()
     and substitute its results for the pending state passed in.

   * Updated sip_session_refresh() with additional debugging.

   * Updated session_reinvite_on_rx_request() to simply return PJ_FALSE
     to pjproject if a transaction is in progress.  This stops us from
     creating a partial pending media state that would be invalid later on.

   * Updated reschedule_reinvite() to clone both the current pending and
     active media states and pass them to delay_request() so the resolver
     can tell what the original intention of the re-invite was.

   * Added a large unit test for the resolver.

ASTERISK-29014

Change-Id: Id3440972943c611a15f652c6c569fa0e4536bfcb

Currently, the ps_contacts table's reg_server column in realtime database type is varchar(20).
This is fine for normal cases, but if the hostname is longer than 20, it returns error and then
failed to register the contact address of the peer.

Normally, 20 characters limitation for the hostname is fine, but with the cloud env.
So, increased the size to 255.

ASTERISK-29056

Change-Id: Iac52c8c35030303cfa551bb39f410b33bffc507d

Currently, it was not possible to create bridge with video_mode single.
This made hard to put the bridge in a vidoe_single mode.
So, added video_single option for Bridge creation using the ARI.
This allows create a bridge with video_mode single.

ASTERISK-29055

Change-Id: I43e720e5c83fc75fafe10fe22808ae7f055da2ae

There's a race condition with bridging where a bridge can be torn down
causing the bridge_channel's ast_channel to become NULL when it's still
needed. This particular case happened with attended transfers, but the
crash occurred when trying to publish a stasis message. Now, the
bridge_channel is locked, a ref to the ast_channel is obtained, and that
ref is passed down the chain.

Change-Id: Ic48715c0c041615d17d286790ae3e8c61bb28814

Building on ASTERISK-25854. When the device requests hold by sending SDP with attribute recvonly, asterisk places the session in sendonly mode. When the device later requests to resume the call by using a re-INVITE excluding SDP, asterisk needs to change the sendonly mode to sendrecv again.

Change-Id: I60341ce3d87f95869f3bc6dc358bd3e8286477a6

Change-Id: Id603b0b03b78eb84c7fca030a08b343c0d5973f9

This fixes a bug introduced mistakenly in ASTERISK-25665:
If leave-empty is enabled, a call may sometimes be removed from
a queue without recording it as abandoned.
This causes Asterisk to not generate an abandon event for that
call, and for the queue abandoned counter to be incorrect.

ASTERISK-29043 #close

Change-Id: I1a71b81df78adff59af587f1d8483cf57df430c7

If you run ast_coredumper --tarball-coredumps in the same directory
as the actual coredump, tar can fail because the link to the
actual coredump becomes recursive.  The resulting tarball will
have everything _except_ the coredump (which is usually what
you need)

There's also an issue that the directory name in the tarball
is the same as the coredump so if you extract the tarball the
directory it creates will overwrite the coredump.

So:

 * Made the link to the coredump use the absolute path to the
   file instead of a relative one.  This prevents the recursive
   link and allows tar to add the coredump.

 * The tarballed directory is now named <coredump>.output instead
   of just <coredump> so if you expand the tarball it won't
   overwrite the coredump.

Change-Id: I8b3eeb26e09a577c702ff966924bb0a2f9a759ea

When fixing issues uncovered by GCC10 a copy of the parker UUID
was removed accidentally. This change restores it so that the
subscription has the data it needs.

ASTERISK-29042

Change-Id: I7d396a14ea648bd26d3c363dd78e78bd386b544a

With the latest Linux, 'ifconfig' is not installed on default anymore.
Furthermore, the output of the current net-tools 'ifconfig' changed.
Therefore, parsing failed. This update uses 'ip addr show' instead.
Finally, the service for the external IP changed.

Change-Id: I9b1a7c3f457e3553b50a3e9a55524e40d70245a0

Since ASTERISK_27978 the default is not off but 90 seconds. That change
happened because ASTERISK_27347 disabled the keep-alives in the bundled
PJProject and Asterisk should behave the same as before.

Change-Id: Ie63dc558ade6a5a2b969c30a4bd492d63730dc46

This patch makes it so if the PJSIP_SEND_SESSION_REFRESH dialplan function
is called on a channel prior to answering a warning is issued and the
function returns unsuccessful.

ASTERISK-28878 #close

Change-Id: I053f767d10cf3b2b898fa9e3e7c35ff07e23c9bb

When the ExtensionState AMI action is executed on a pattern matched
hint it can end up adding a new hint if one does not already exist.
This results in a locking order of contexts -> hints -> contexts.

If at the same time a reload is occurring and adding its own hint
it will have a locking order of hints -> contexts.

This results in a deadlock as one thread wants a lock on contexts
that the other has, and the other thread wants a lock on hints
that the other has.

This change enforces a hints -> contexts locking order by explicitly
locking hints in the places where a hint is added when queried for.
This matches the order seen through normal adding of hints.

ASTERISK-29046

Change-Id: I49f027f4aab5d2d50855ae937bcf5e2fd8bfc504

Added a new log formatter called "plain" that always prints
file, function and line number if available (even for verbose
messages) and never prints color control characters.  It also
doesn't apply any special formatting for verbose messages.
Most suitable for file output but can be used for other channels
as well.

You use it in logger.conf like so:
debug => [plain]debug
console => [plain]error,warning,debug,notice,pjsip_history
messages => [plain]warning,error,verbose

Change-Id: I4fdfe4089f66ce2f9cb29f3005522090dbb5243d

Properly bump reference on format object to avoid memory corruption on double free

ASTERISK-29040 #close

Change-Id: Ic5a7faabfe2ef965ddb024186e1de7ca4542e2a3

Adapt the response handler so it also called when 181 is received.
In the case 181 is received, also generate the 181 response.

ASTERISK-29001 #close

Change-Id: I73cfee46a8ca85371280ebdb38674f8fde7510df

Rather than putting messages into INBOX and then moving them to Urgent
later, put them directly in to the Urgent folder. This prevents
mailcmd from being skipped.

ASTERISK-27273 #close

Change-Id: I49934e093290d308506ab8d45a40ef705c5ae4f5

This fixes the reseting members lastpause problem when realtime members is being used,
the function rt_handle_member_record was forcing the reset members lastpause because it
does not exist in realtime

ASTERISK-29034 #close

Change-Id: Ic9107e4456732a1f78412a32adb2ef87f5da40b5

Per the RFC when an outgoing re-INVITE is done we should
only terminate the dialog if a 481 or 408 is received.

ASTERISK-29033

Change-Id: I6c3ff513aa41005d02de0396ba820083e9b18503

T.140 data in RTP is not zero terminated, so when we are queuing a text
frame on a bridge we need to ensure that we are passing a zero
terminated string.

ASTERISK-28974 #close

Change-Id: Ic10057387ce30b2094613ea67e3ae8c5c431dda3

Two changes of note in this patch:

* Use ast_file_read_dir instead of opendir/readdir/closedir

* If the files list should be sorted, do that at the end rather than as
  we go which improves performance for large lists

Change-Id: Ic7e9c913c0f85754c99c74c9cf6dd3514b1b941f

The SCOPE_ENTER and SCOPE_EXIT* macros now print debug messages
at the same level as the scope level.  This allows the same
messages to be printed to the debug log when AST_DEVMODE
isn't enabled.

Also added a few variants of the SCOPE_EXIT macros that will
also call ast_log instead of ast_debug to make it easier to
use scope tracing and still print error messages.

Change-Id: I7fe55f7ec28069919a0fc0b11a82235ce904cc21

 * Added ast_stream_to_stra and ast_stream_topology_to_stra() macros
   which are shortcuts for
      ast_str_tmp(256, ast_stream_to_str(stream, &STR_TMP))

 * Added the stream position to the string representation of the
   stream.

 * Fixed some formatting in ast_stream_to_str().

Change-Id: Idaf4cb0affa46d4dce58a73a111f35435331cc4b

The ToHost parameter was not cleared when a peer's host value was
changed to dynamic. This causes invites to be sent to the original host.

ASTERISK-29011 #close

Change-Id: I9678d512741f71baca8f131a65b7523020b07d5c

Allow passing a topology from the called channel back to the
calling channel.

 * Added a new function ast_queue_answer() that accepts a stream
   topology and queues an ANSWER CONTROL frame with it as the
   data.  This allows the called channel to indicate its resolved
   topology.

 * Added a new virtual function to the channel tech structure
   answer_with_stream_topology() that allows the calling channel
   to receive the called channel's topology.  Added
   ast_raw_answer_with_stream_topology() that invokes that virtual
   function.

 * Modified app_dial.c and features.c to grab the topology from the
   ANSWER frame queued by the answering channel and send it to
   the calling channel with ast_raw_answer_with_stream_topology().

 * Modified frame.c to automatically cleanup the reference
   to the topology on ANSWER frames.

Added a few debugging messages to stream.c.

Change-Id: I0115d2ed68d6bae0f87e85abcf16c771bdaf992c

Adds sed before awk to produce reasonable ASTERISKVERSIONNUM
on certified versions of Asterisk eg. 16.8-cert3 is 160803
instead of the previous 00800.

ASTERISK-29021 #close

Change-Id: Icf241df0ff6db09011b8c936a317a84b0b634e16

The MoH class internal file vector is potentially being manipulated by
multiple threads at the same time without sufficient locking. Switch to
a reference counted list and operate on copies where necessary.

ASTERISK-28927 #close

Change-Id: I479c5dcf88db670956e8cac177b5826c986b0217

When reading in a codec preference configuration option
the value would be set on the respective option before
applying any default adjustments, resulting in the
configuration not being as expected.

This was exposed by the REST API push configuration as
it used the configuration returned by Asterisk to then do
a modification. In the case of codec preferences one of
the options had a transcode value of "unspecified" when the
defaults should have ensured it would be "allow" instead.

This also renames the options in other places that were
missed.

Change-Id: I4ad42e74fdf181be2e17bc75901c62591d403964

The assumed behavior of realloc() - that it was effectively a free() if
its second argument was 0 - is Linux specific behavior and is not
guaranteed by either POSIX or the C specification.

Instead, if we want to resize a vector to 0, do it explicitly.

Change-Id: Ife31d4b510ebab41cb5477fdc7ea4e3138ca8b4f

PJSIP, UDP transport with external_media_address and session timers
enabled. Connected to SIP server that is not in local net. Asterisk
initiated the connection and is refreshing the session after 150s
(timeout 300s). The 2nd refresh-INVITE triggered by the pjsip timer has
a malformed IP address in its SDP (garbage string). This only happens
when the SDP is modified by the nat-code to replace the local IP address
with the configured external_media_address.
Analysis: the code to modify the SDP (in
res_pjsip_session.c:session_outgoing_nat_hook() and also (redundantly?)
in res_pjsip_sdp_rtp.c:change_outgoing_sdp_stream_media_address()) uses
the tdata->pool to allocate the replacement string. But the same
pjmedia_sdp_stream that was modified for the 1st refresh-INVITE is also
used for the 2nd refresh-INVITE (because it is stored in pjmedia's
pjmedia_sdp_neg structure). The problem is, that at that moment, the
tdata->pool that holds the stringified external_media_address from the
1. refresh-INVITE has long been reused for something else.
Fix by Sauw Ming of pjproject (see
https://github.com/pjsip/pjproject/pull/2476): the local, potentially
modified pjmedia_sdp_stream is cloned in
pjproject/source/pjsip/src/pjmedia/sip_neg.c:process_answer() and the
clone is stored, thereby detaching from the tdata->pool (which is only
released *after* process_answer())

ASTERISK-28973
Reported-by: Michael Neuhauser

Change-Id: I272ac22436076596e06aa51b9fa23fd1c7734a0e

With the addition of STIR/SHAKEN, the function ast_base64decode_string
was added for convenience since there is a lot of converting done during
the STIR/SHAKEN process. This function returned the decoded string for
you, but did not NULL terminate it, causing some issues (specifically
with MALLOC_DEBUG). Now, the returned string is NULL terminated, and the
documentation has been updated to reflect this.

Change-Id: Icdd7d05b323b0c47ff6ed43492937a03641bdcf5

This change renames the codec preference endpoint options.
incoming_offer_codec_prefs becomes codec_prefs_incoming_offer
to keep the options together when showing an endpoint.

Change-Id: I6202965b4723777f22a83afcbbafcdafb1d11c8d

Fixed a memory allocation that was not passing in the correct size for
the struct in curl.c.

Change-Id: I5fb92fbbe84b075fa6aefa2423786df80e114c3a

When a bundled stream is removed, its bundle_group is reset to -1.
If that stream is later reused, the bundle parameters on session
media need to be reset correctly it could mistakenly be rebundled
with a stream that was removed and never reused.  Since the removed
stream has no rtp instance, a crash will result.

Change-Id: Ie2b792220f9291587ab5f9fd123145559dba96d7

Statically configured contacts on an AOR don't have an expiration
time so when adding them to the resulting 200 OK if an endpoint
registers ensure they are marked as such.

ASTERISK-28995

Change-Id: I9f0e45eb2ccdedc9a0df5358634a19ccab0ad596

There are various places in Asterisk - specifically in regards to
database integration - where having some kind of UTF-8 validation would
be beneficial. This patch adds:

* Functions to validate that a given string contains only valid UTF-8
  sequences.

* A function to copy a string (similar to ast_copy_string) stopping when
  an invalid UTF-8 sequence is encountered.

* A UTF-8 validator that allows for progressive validation.

All of this is based on the excellent UTF-8 decoder by Björn Höhrmann.
More information is available here:

    https://bjoern.hoehrmann.de/utf-8/decoder/dfa/

The API was written in such a way that should allow us to replace the
implementation later should we determine that we need something more
comprehensive.

Change-Id: I3555d787a79e7c780a7800cd26e0b5056368abf9

Currently, if the bridge has created by the ARI, the video_mode
parameter was
not shown in the BridgeCreated event correctly.

Fixed it and added video_mode shown in the 'bridge show <bridge id>'
cli.

ASTERISK-28987

Change-Id: I8c205126724e34c2bdab9380f523eb62478e4295