There are two main parts of the change associated with this
commit. These are driven by the change in call order of
pubsub_on_rx_refresh and pubsub_on_evsub_state by pjproject
when an in-dialog SUBSCRIBE is received.

First, the previous behavior was for pjproject to call
pubsub_on_rx_refresh before calling pubsub_on_evsub_state
when an in-dialog SUBSCRIBE was received that changes the
subscription state.

If that change was a termination due to a re-SUBSCRIBE with
an expires of 0, we used to use the call to pubsub_on_rx_refresh
to set the substate of the evsub to TERMINATE_PENDING before
pjproject could call pubsub_on_evsub_state.

This substate let pubsub_on_evsub_state know that the
subscription TERMINATED event could be ignored as there was
still a subsequent NOTIFY that needed to be generated and
another call to pubsub_on_evsub_state to come with it.

That NOTIFY was sent via serialized_pubsub_on_refresh_timeout
which would see the TERMINATE_PENDING state and transition it
to TERMINATE_IN_PROGRESS before triggering another call to
pubsub_on_evsub_state (which now would clean up the evsub.)

The new pjproject behavior is to call pubsub_on_evsub_state
before pubsub_on_rx_refresh. This means we no longer can set
the state to TERMINATE_PENDING to tell pubsub_on_evsub_state
that it can ignore the first TERMINATED event.

To handle this, we now look directly at the event type,
method type and the expires value to determine whether we
want to ignore the event or use it to trigger the evsub
cleanup.

Second, pjproject now expects the NOTIFY to actually be sent
during pubsub_on_rx_refresh and avoids the protocol violation
inherent in sending a NOTIFY before the SUBSCRIBE is
acknowledged by caching the sent NOTIFY then sending it
after responding to the SUBSCRIBE.

This requires we send the NOTIFY using the non-serialized
pubsub_on_refresh_timeout directly and let pjproject handle
the protocol violation.

ASTERISK-30469

Change-Id: I05c1d91a44fe28244ae93faa4a2268a3332b5fd7

This reverts commit 56051d1a.

Reason for revert: Behavior change that breaks existing dialplan.

ASTERISK-30472 #close

Change-Id: I83bed3b800d36228a04ded0a6164b795f7f16bd6

* Added a new function ast_utf8_replace_invalid_chars() to
  utf8.c that copies a string replacing any invalid UTF-8
  sequences with the Unicode specified U+FFFD replacement
  character.  For example:  "abc\xffdef" becomes "abc\uFFFDdef".
  Any UTF-8 compliant implementation will show that character
  as a � character.

* Updated res_pjsip:set_id_from_hdr() to use
  ast_utf8_replace_invalid_chars and print a warning if any
  invalid sequences were found during the copy.

* Updated stasis_channels:ast_channel_publish_varset to use
  ast_utf8_replace_invalid_chars and print a warning if any
  invalid sequences were found during the copy.

ASTERISK-27830

Change-Id: I4ffbdb19c80bf0efc675d40078a3ca4f85c567d8

Phones moving between subnets on multi-homed server have their
initially connected interface IP cached in the SERVER variable,
even when it is not specified in the configuration files. This
prevents phones from obtaining the correct SERVER variable value
when they move to another subnet.

ASTERISK-30388 #close
Reported-by: cmaj

Change-Id: I1d18987a9d58e85556b4c4a6814ce7006524cc92

contributed pjproject - patch to check sub->pending_notify
in evsub.c:on_tsx_state before calling
pjsip_evsub_send_request()

res_pjsip_pubsub - change post pjsip 2.13 behavior to use
pubsub_on_refresh_timeout to avoid the ao2_cleanup call on
the sub_tree. This is is because the final NOTIFY send is no
longer the last place the sub_tree is referenced.

ASTERISK-30419

Change-Id: Ib5cc662ce578e9adcda312e16c58a10b6453e438

Removed multiple patches.

Code chages in res_pjsip_pubsub due to changes in evsub.

Pjsip now calls on_evsub_state() before on_rx_refresh(),
so the sub tree deletion that used to take place in
on_evsub_state() now must take place in on_rx_refresh().

Additionally, pjsip now requires that you send the NOTIFY
from within on_rx_refresh(), otherwise it will assert
when going to send the 200 OK. The idea is that it will
look for this NOTIFY and cache it until after sending the
response in order to deal with the self-imposed message
mis-order. Asterisk previously dealt with this by pushing
the NOTIFY in on_rx_refresh(), but pjsip now forces us
to use it's method.

Changes were required to configure in order to detect
which way pjsip handles this as the two are not
compatible for the reasons mentioned above.

A corresponding change in testsuite is required in order
to deal with the small interal timing changes caused by
moving the NOTIFY send.

ASTERISK-30325

Change-Id: I50b00cac89d950d3511d7b250a1c641965d9fe7f

Change-Id: Ic50e95b4fc10f74ab15416d908e8a87ee8ec2f85

Added NULL pointer check and channel lock to prevent resource release
while the chanspy is processing.

ASTERISK-29604

Change-Id: Ibdc675f98052da32333b19685b1708a3751b6d24

Variable references within global variable assignments are now
expanded rather than being included literally.

ASTERISK-30406 #close

Change-Id: I136e8d6395e90a4c92d9777a46a7bc3edb08d05d

Adds the overlap_context option, which can be used
to explicitly specify a context to use for overlap
dialing extension matches, rather than forcibly
using the context configured for the endpoint.

ASTERISK-30262 #close

Change-Id: Ibbcd4a8b11402428a187fb56b8d4e7408774a0db

Rounding issues with double math were causing rtp timestamp
slips in outgoing packets.  We're now back to integer math
and are getting no more slips.

ASTERISK-30391

Change-Id: I6ba992b49ffdf9ebea074581dfa784a188c661a4

Add ability to set HANGUPCAUSE when SIP causecode received in BYE (in addition to currently supported Q.850).

ASTERISK-30319 #close

Change-Id: I3f55622dc680ce713a2ffb5a458ef5dd39fcf645

-----------------

This commit reinstates MES with some casting fixes to the
functions in time.h that convert between doubles and timeval
structures.  The casting issues were causing incorrect
timestamps to be calculated which caused transcoding from/to
G722 to produce bad or no audio.

ASTERISK-30391

-----------------

This module has been updated to provide additional
quality statistics in the form of an Asterisk
Media Experience Score.  The score is avilable using
the same mechanisms you'd use to retrieve jitter, loss,
and rtt statistics.  For more information about the
score and how to retrieve it, see
https://wiki.asterisk.org/wiki/display/AST/Media+Experience+Score

* Updated chan_pjsip to set quality channel variables when a
  call ends.
* Updated channels/pjsip/dialplan_functions.c to add the ability
  to retrieve the MES along with the existing rtcp stats when
  using the CHANNEL dialplan function.
* Added the ast_debug_rtp_is_allowed and ast_debug_rtcp_is_allowed
  checks for debugging purposes.
* Added several function to time.h for manipulating time-in-samples
  and times represented as double seconds.
* Updated rtp_engine.c to pass through the MES when stats are
  requested.  Also debug output that dumps the stats when an
  rtp instance is destroyed.
* Updated res_rtp_asterisk.c to implement the calculation of the
  MES.  In the process, also had to update the calculation of
  jitter.  Many debugging statements were also changed to be
  more informative.
* Added a unit test for internal testing.  The test should not be
  run during normal operation and is disabled by default.

Change-Id: I4fce265965e68c3fdfeca55e614371ee69c65038

This reverts commit d454801c.

Reason for revert: Issue when transcoding to/from g722

Change-Id: I09f49e171b1661548657a9ba7a978c29d0b5be86

When Asterisk receives a new websocket conenction, it creates a new
pjsip transport for it and copies connection data into it.  The
transport manager then uses the remote IP address and port on the
transport to create a monitor for each connection.  However, the
remote port wasn't being copied, only the IP address which meant
that the transport manager was creating only 1 monitoring entry for
all websocket connections from the same IP address. Therefore, if
one of those connections failed, it deleted the transport taking
all the the connections from that same IP address with it.

* We now copy the remote port into the created transport and the
  transport manager behaves correctly.

ASTERISK-30369

Change-Id: Ib506d40897ea6286455ac0be4dfbb0ed43b727e1

Do not crash when a URL has no path component as in this case the
ast_uri_path function will return NULL. Make the code cope with not
having a path.

The below would crash
> media cache create http://google.com /tmp/foo.wav

Thread 1 "asterisk" received signal SIGSEGV, Segmentation fault.
0x0000ffff836616cc in strrchr () from /lib/aarch64-linux-gnu/libc.so.6
(gdb) bt
 #0  0x0000ffff836616cc in strrchr () from /lib/aarch64-linux-gnu/libc.so.6
 #1  0x0000ffff43d43a78 in file_extension_from_string (str=<optimized out>, buffer=buffer@entry=0xffffca9973c0 "",
    capacity=capacity@entry=64) at res_http_media_cache.c:288
 #2  0x0000ffff43d43bac in file_extension_from_url_path (bucket_file=bucket_file@entry=0x3bf96568,
    buffer=buffer@entry=0xffffca9973c0 "", capacity=capacity@entry=64) at res_http_media_cache.c:378
 #3  0x0000ffff43d43c74 in bucket_file_set_extension (bucket_file=bucket_file@entry=0x3bf96568) at res_http_media_cache.c:392
 #4  0x0000ffff43d43d10 in bucket_file_run_curl (bucket_file=0x3bf96568) at res_http_media_cache.c:555
 #5  0x0000ffff43d43f74 in bucket_http_wizard_create (sorcery=<optimized out>, data=<optimized out>, object=<optimized out>)
    at res_http_media_cache.c:613
 #6  0x0000000000487638 in bucket_file_wizard_create (sorcery=<optimized out>, data=<optimized out>, object=<optimized out>)
    at bucket.c:191
 #7  0x0000000000554408 in sorcery_wizard_create (object_wizard=object_wizard@entry=0x3b9f0718,
    details=details@entry=0xffffca9974a8) at sorcery.c:2027
 #8  0x0000000000559698 in ast_sorcery_create (sorcery=<optimized out>, object=object@entry=0x3bf96568) at sorcery.c:2077
 #9  0x00000000004893a4 in ast_bucket_file_create (file=file@entry=0x3bf96568) at bucket.c:727
 #10 0x00000000004f877c in ast_media_cache_create_or_update (uri=0x3bfa1103 "https://google.com",
    file_path=0x3bfa1116 "/tmp/foo.wav", metadata=metadata@entry=0x0) at media_cache.c:335
 #11 0x00000000004f88ec in media_cache_handle_create_item (e=<optimized out>, cmd=<optimized out>, a=0xffffca9976b8)
    at media_cache.c:640

ASTERISK-30375 #close

Change-Id: I6a9433688cb5d3d4be8758b7642d923bdde6c273

This module has been updated to provide additional
quality statistics in the form of an Asterisk
Media Experience Score.  The score is avilable using
the same mechanisms you'd use to retrieve jitter, loss,
and rtt statistics.  For more information about the
score and how to retrieve it, see
https://wiki.asterisk.org/wiki/display/AST/Media+Experience+Score

* Updated chan_pjsip to set quality channel variables when a
  call ends.
* Updated channels/pjsip/dialplan_functions.c to add the ability
  to retrieve the MES along with the existing rtcp stats when
  using the CHANNEL dialplan function.
* Added the ast_debug_rtp_is_allowed and ast_debug_rtcp_is_allowed
  checks for debugging purposes.
* Added several function to time.h for manipulating time-in-samples
  and times represented as double seconds.
* Updated rtp_engine.c to pass through the MES when stats are
  requested.  Also debug output that dumps the stats when an
  rtp instance is destroyed.
* Updated res_rtp_asterisk.c to implement the calculation of the
  MES.  In the process, also had to update the calculation of
  jitter.  Many debugging statements were also changed to be
  more informative.
* Added a unit test for internal testing.  The test should not be
  run during normal operation and is disabled by default.

ASTERISK-30280

Change-Id: I458cb9a311e8e5dc1db769b8babbcf2e093f107a

Currently, there is no Caller ID available to us when
checking for an extension match when handling INVITEs.
As a result, extension patterns that depend on the Caller ID
are not matched and calls may be incorrectly rejected.

The Caller ID is not available because the supplement that
adds Caller ID to the session does not execute until after
this check. Supplement callbacks cannot yet be executed
at this point since the session is not yet in the appropriate
state.

To fix this without impacting existing behavior, the Caller ID
number is now retrieved before attempting to pattern match.
This ensures pattern matching works correctly and there is
no behavior change to the way supplements are called.

ASTERISK-28767 #close

Change-Id: Iec7f5a3b90e51b65ccf74342f96bf80314b7cfc7

When a call is put on hold and it has moh_passthrough and rtp_timeout
set on the endpoint, the wrong timeout will be used. rtp_timeout_hold is
expected to be used, but rtp_timeout is used instead. This change adds a
couple of checks for locally_held to determine if rtp_timeout_hold needs
to be used instead of rtp_timeout.

ASTERISK-30350

Change-Id: I7b106fc244332014216d12bba851cefe884cc25f

Fix aor lookup on sip path addition. Issue happens in case of dialing
with @ and overriding user part of RURI.

ASTERISK-30100 #close
Reported-by: Yury Kirsanov

Change-Id: I3f2c42a583578c94397b113e32ca3ebf2d600e13

The `ast_geoloc_datastore_add_eprofile` function does not return 0 on
success, it returns the size of the underlying datastore. This means
that the datastore will be freed and its pointer set to NULL when no
error occured at all.

ASTERISK-30346

Change-Id: Iea9b209bd1244cc57b903b9496cb680c356e4bb9

When adding AOC to an outgoing response the code
assumed that a body would exist for comparing the
Content-Type. This isn't always true.

The code now checks to make sure the response has
a body before checking the Content-Type.

ASTERISK-21502

Change-Id: Iaead371434fc3bc693dad487228106a7d7a5ac76

chan_sip supported sending AOC-D and AOC-E information in SIP INFO
messages in an "AOC" header in a format that was originally defined by
Snom. In the meantime, ETSI TS 124 647 introduced an XML-based AOC
format that is supported by devices from multiple vendors, including
Snom phones with firmware >= 8.4.2 (released in 2010).

This commit adds a new res_pjsip_aoc module that inserts AOC information
into outgoing messages or sends SIP INFO messages as described below.
It also fixes a small issue in res_pjsip_session which didn't always
call session supplements on outgoing_response.

* AOC-S in the 180/183/200 responses to an INVITE request
* AOC-S in SIP INFO (if a 200 response has already been sent or if the
  INVITE was sent by Asterisk)
* AOC-D in SIP INFO
* AOC-D in the 200 response to a BYE request (if the client hangs up)
* AOC-D in a BYE request (if Asterisk hangs up)
* AOC-E in the 200 response to a BYE request (if the client hangs up)
* AOC-E in a BYE request (if Asterisk hangs up)

The specification defines one more, AOC-S in an INVITE request, which
is not implemented here because it is not currently possible in
Asterisk to have AOC data ready at this point in call setup. Once
specifying AOC-S via the dialplan or passing it through from another
SIP channel's INVITE is possible, that might be added.

The SIP INFO requests are sent out immediately when the AOC indication
is received. The others are inserted into an appropriate outgoing
message whenever that is ready to be sent. In the latter case, the XML
is stored in a channel variable at the time the AOC indication is
received. Depending on where the AOC indications are coming from (e.g.
PRI or AMI), it may not always be possible to guarantee that the AOC-E
is available in time for the BYE.

Successfully tested AOC-D and both variants of AOC-E with a Snom D735
running firmware 10.1.127.10. It does not appear to properly support
AOC-S however, so that could only be tested by inspecting SIP traces.

ASTERISK-21502 #close
Reported-by: Matt Jordan <mjordan@digium.com>

Change-Id: Iebb7ad0d5f88526bc6629d3a1f9f11665434d333

When passing a JSON body to the 'create' channel route
it would be converted into Asterisk variables, but never
freed resulting in a memory leak.

This change makes it so that the variables are freed in
all cases.

ASTERISK-30344

Change-Id: I924dbd866a01c6073e2d6fb846ccaa27ef72d49d

This fixes a small typo in the from_domain documentation on the endpoint documentation

ASTERISK-30328 #close

Change-Id: Ia6f0897c3f5cab899ef2cde6b3ac07265b8beb21

Adds support for the capture agent name field
of the Homer protocol to Asterisk by allowing
users to specify a name that will be sent to
the HEP server.

ASTERISK-30322 #close

Change-Id: I6136583017f9dd08daeb8be02f60fb8df4639a2b

Some SIP devices use an empty extension for PLAR functionality.

Rather than rejecting these empty extensions, we now use the s
extension for such calls to mirror the existing PLAR functionality
in Asterisk (e.g. chan_dahdi).

ASTERISK-30265 #close

Change-Id: I0861a405cd49bbbf532b52f7b47f0e2810832590

Updates the documentation for the 'contact_user' field to point out the
default outbound contact if no contact_user is specified 's'

ASTERISK-30316 #close

Change-Id: I61f24fb9164e4d07e05908a2511805281874c876

The commit that rearchitected media formats,
a2c912e9 (ASTERISK_23114)
introduced a regression by improperly translating code in res_adsi.c.
In particular, the pointer to the frame buffer was initialized
at the top of adsi_careful_send, rather than dynamically updating it
for each frame, as is required.

This resulted in the first frame being repeatedly sent,
rather than advancing through the frames.
This corrupted the transmission of the CAS to the CPE,
which meant that CPE would never respond with the DTMF acknowledgment,
effectively completely breaking ADSI functionality.

This issue is now fixed, and ADSI now works properly again.

ASTERISK-29793 #close

Change-Id: Icdeddf733eda2981c98712d1ac9cddc0db507dbe

Adds support for custom URI and header parameters
in the From header in PJSIP. Parameters can be
both set and read using this function.

ASTERISK-30150 #close

Change-Id: Ifb1bc3c512ad5f6faeaebd7817f004a2ecbd6428

It was possible for a module that registered for transport monitor
events to pass in a pjsip_transport that had already been freed.
This caused pjsip_transport_events to crash when looking up the
monitor for the transport.  The fix is a two pronged approach.

1. We now increment the reference count on pjsip_transports when we
create monitors for them, then decrement the count when the
transport is going to be destroyed.

2. There are now APIs to register and unregister monitor callbacks
by "transport key" which is a string concatenation of the remote ip
address and port.  This way the module needing to monitor the
transport doesn't have to hold on to the transport object itself to
unregister.  It just has to save the transport_key.

* Added the pjsip_transport reference increment and decrement.

* Changed the internal transport monitor container key from the
  transport->obj_name (which may not be unique anyway) to the
  transport_key.

* Added a helper macro AST_SIP_MAKE_REMOTE_IPADDR_PORT_STR() that
  fills a buffer with the transport_key using a passed-in
  pjsip_transport.

* Added the following functions:
  ast_sip_transport_monitor_register_key
  ast_sip_transport_monitor_register_replace_key
  ast_sip_transport_monitor_unregister_key
  and marked their non-key counterparts as deprecated.

* Updated res_pjsip_pubsub and res_pjsip_outbound_register to use
  the new "key" monitor functions.

NOTE: res_pjsip_registrar also uses the transport monitor
functionality but doesn't have a persistent object other than
contact to store a transport key.  At this time, it continues to
use the non-key monitor functions.

ASTERISK-30244

Change-Id: I1a20baf2a8643c272dcf819871d6c395f148f00b

This PR contains two relatively separate changes in channel.c and
res_pjsip_session.c which ensure that topology changes are not ignored
in cases where they should be handled.

For channel.c:

The function ast_channel_request_stream_topology_change only triggers a
stream topology request change indication, if the channel's topology
does not equal the requested topology. However, a channel could be in a
state where it is currently "negotiating" a new topology but hasn't
updated it yet, so the topology request change would be lost. Channels
need to be able to handle such situations internally and stream
topology requests should therefore always be passed on.

In the case of chan_pjsip for example, it queues a session refresh
(re-INVITE) if it is currently in the middle of a transaction or has
pending requests (among other reasons).

Now, ast_channel_request_stream_topology_change always indicates a
stream topology request change even if the requested topology equals the
channel's topology.

For res_pjsip_session.c:

The function resolve_refresh_media_states does not process stream state
changes if the delayed active state differs from the current active
state. I.e. if the currently active stream state has changed between the
time the sip session refresh request was queued and the time it is being
processed, the session refresh is ignored. However, res_pjsip_session
contains logic that ensures that session refreshes are queued and
re-queued correctly if a session refresh is currently not possible. So
this check is not necessary and led to some session refreshes being
lost.

Now, a session refresh is done even if the delayed active state differs
from the current active state and it is checked whether the delayed
pending state differs from the current active - because that means a
refresh is necessary.

Further, the unit test of resolve_refresh_media_states was adapted to
reflect the new behavior. I.e. the changes to delayed pending are
prioritized over the changes to current active because we want to
preserve the original intention of the pending state.

ASTERISK-30184

Change-Id: Icd0703295271089057717006730b555b9a1d4e5a

The "RECORD FILE" command in res_agi has its own
implementation for actually doing the recording. This
has resulted in it not actually obeying the option
"transmit_silence" when recording.

This change causes it to now send silence if the
option is enabled.

ASTERISK-30314

Change-Id: Ib3a85601ff35d1b904f836691bad8a4b7e957174

When a websocket (or potentially any stateful connection) is quickly
created then destroyed, it is possible that the qualify thread will
destroy the transaction before the initialzing thread is finished
with it.

Depending on the timing, this can cause an assertion within pjsip.

To prevent this, ast_send_stateful_response will now create the group
lock and add a reference to it before creating the transaction.

While this should resolve the crash, there is still the potential that
the contact will not be cleaned up properly, see:ASTERISK~29286. As a
result, the contact has to 'time out' before it will be removed.

ASTERISK-28689

Change-Id: Id050fded2247a04d8f0fc5b8a2cf3e5482cb8cee

Current registration code use pjsip_parse_uri to verify outbound_proxy
that is different from the reading this option for the endpoint. This
made value with multiple proxies invalid for registration pjsip settings.
Removing URI validation helps to use registration through multiple proxies.

ASTERISK-30217 #close

Change-Id: I064558e66f04b9f3260c46181812a01349761357

Fix compilation errors caused by using size_t
instead of uintmax_t and non-portable format
specifiers.

ASTERISK-30273 #close

Change-Id: I363e6057ef84d54b88af80d23ad6147eef9216ee

Currently chan_pjsip on receiving a re-INVITE without SDP will only
return the codecs that are previously negotiated and not offering
all enabled codecs.

This causes interoperability issues with different equipment (e.g.
from Cisco) for some of our customers and probably also in other
scenarios involving 3PCC infrastructure.

According to RFC 3261, section 14.2 we SHOULD return all codecs
on a re-INVITE without SDP

The PR proposes a new parameter to configure this behaviour:
all_codecs_on_empty_reinvite. It includes the code, documentation,
alembic migrations, CHANGES file and example configuration additions.

ASTERISK-30193 #close

Change-Id: I69763708d5039d512f391e296ee8a4d43a1e2148

The PJSIP notify CLI commands allow for using
"options" configured in pjsip_notify.conf.

This allows these same options to be used in
AMI actions as well.

Additionally, as part of this improvement,
some repetitive common code is refactored.

ASTERISK-30263 #close

Change-Id: Ie4496b322b63b61eaf9672183a959ab99a04b6b5

Expands the pjsip logger to support the ability to filter
by SIP message method. This can make certain types of SIP debugging
easier by only logging messages of particular method(s).

ASTERISK-30146 #close

Co-authored-by: Sean Bright <sean@seanbright.com>
Change-Id: I9c8cbb6fc8686ef21190eb42e08bc9a9b147707f

pjproject does not provide any mechanism of removing
event packages, which means that once a subscription
handler is registered, it is effectively permanent.

pjproject will assert if the same event package is
ever registered again, so currently unloading and
loading any Asterisk modules that use subscriptions
will cause a crash that is beyond our control.

For that reason, we now prevent users from being
able to unload these modules, to prevent them
from ever being loaded twice.

ASTERISK-30264 #close

Change-Id: I7fdcb1a5e44d38b7ba10c44259fe98f0ae9bc12c