CLI command 'pjsip show contacts' inefficiently make a lot of DB requests.

For example if there are 10k aors then asterisk requests these 10k records
of aor and then does 10k requests of contact - one request per aor.

Even if use 'like <pattern>' the asterisk requests all aor's and contact's
records and then filters them by itself.

This patch gathers contact's container by
- retrieving all dynamic contacts by regex (filtered by reg_server)
- retrieving all aors with permanent contacts
- finally filters container by regex

ASTERISK-28077 #close

Change-Id: Id0ad65d14952a02fb213273a90f3f680a8149618

Fixed an issue that resulted in "Allocation failed" each time an ARI
request was made to start playing MOH on a bridge.

In bridge_moh_create() we were attaching the after bridge callbacks to
chan which is the ;1 channel of the unreal channel pair.  We should have
attached them to the ;2 channel which is pushed into the bridge by
ast_unreal_channel_push_to_bridge().  The callbacks are called when the
specific channel leaves the bridging system.  Since the ;1 channel is
never put into a bridge the callbacks never get called.  The callbacks
then never remove the moh_wrapper from the app_bridges_moh container.  As
a result we cannot find the channel associated with the wrapper to start
MOH because it has hungup.  This is the reason causing the reported issue.

* Rather than using after bridge callbacks to cleanup, we now have
moh_channel_thread() doing the cleanup when the channel hangs up.

* Fixed moh_channel_thread() accumulating control frames on the stasis
bridge MOH channel until MOH is stopped.  Control frames are no longer
accumulated while MOH is playing.

* Fixed channel ref counting issue.  stasis_app_bridge_moh_channel() may
or may not return a channel ref.  As a result ast_ari_bridges_start_moh()
wouldn't know it may have a channel ref to release.
stasis_app_bridge_moh_channel() will now return a ref with the channel it
returns.

* Eliminated RAII_VAR in bridge_moh_create().

ASTERISK-26094 #close

Change-Id: Ibff479e167b3320c68aaabfada7e1d0ef7bd548c

When networks experience disruptions, there can be large gaps of time
between receiving packets. When strictrtp is enabled, this created
issues where a flood of packets could come in and be seen as an attack.
Another option - seqno - has been added to the strictrtp option that
ignores the time interval and goes strictly by sequence number for
validity.

Change-Id: I8a42b8d193673899c8fc22fe7f98ea87df89be71

On SQL error there is not diagnostic information about this error.
There is only
WARNING res_odbc.c: SQL Execute error -1!

The function ast_odbc_print_errors calls a SQLGetDiagField to get the number
of available diagnostic records, but the SQLGetDiagField returns 0.
However SQLGetDiagRec could return one diagnostic records in this case.

Looking at many example of getting diagnostics error information
I found out that the best way it's to use only SQLGetDiagRec
while it returns SQL_SUCCESS.

Also this patch adds calls of ast_odbc_print_errors on SQL_ERROR
to res_config_odbc.

ASTERISK-28065 #close

Change-Id: Iba5ae5470ac49ecd911dd084effbe9efac68ccc1

In order to do this and provide good feedback, a new macro was
created (AST_EXT_LIB_EXTRA_CHECK) which does the normal check and
path setups for the library then compiles, links and runs a supplied
code fragment to do the final determination.  In this case, the
final code fragment compares UNBOUND_VERSION_MAJOR
and UNBOUND_VERSION_MINOR to determine if they're greater than or
equal to 1.5.

Since we require version 1.5, some code in res_resolver_unbound
was also simplified.

ASTERISK-28045
Reported by: Samuel Galarneau

Change-Id: Iee94ad543cd6f8b118df8c4c7afd9c4e2ca1fa72

This change raises a testsuite event to provide what port
Asterisk has actually allocated for RTP. This ensures that
testsuite tests can remove any assumption of ports and instead
use the actual port in use.

ASTERISK-28070

Change-Id: I91bd45782e84284e01c89acf4b2da352e14ae044

* Use "o*" format specifier for optional fields in ast_json_party_id.
* Stop using ast_json_deep_copy on immutable objects, it is now thread
  safe to just use ast_json_ref.

Additional changes to ast_json_pack calls in the vicinity:
* Use "O" when an object needs to be bumped.  This was previously
  avoided as it was not thread safe.
* Use "o?" and "O?" to replace NULL with ast_json_null().  The
  "?" is a new feature of ast_json_pack starting with Asterisk 16.

Change-Id: I8382d28d7d83ee0ce13334e51ae45dbc0bdaef48

ast_rtp_new free'd rtp upon failure, but rtp_engine.c would also call
the destroy callback.  Remove call to ast_free from ast_rtp_new, leave
it to rtp_engine.c to initiate the full cleanup.  Add error detection
for the ssrc_mapping vector initialization.  In rtp_allocate_transport
set rtp->s = -1 in the failure path where we close that FD to ensure we
don't try closing it twice.

ASTERISK-27854 #close

Change-Id: Ie02aecbb46228ca804e24b19cec2bb6f7b94e451

'rtpchecksums' and 'rtcpinterval' are not being reset to their defaults
if they are not present in the updated configuration file.

Change-Id: I1162e40199314d46cf3225d5e1271c4c81176670

The HTTP request processing in res_http_websocket allocates additional
space on the stack for various headers received during an Upgrade request.
An attacker could send a specially crafted request that causes this code
to overflow the stack, resulting in a crash.

* No longer allocate memory from the stack in a loop to parse the header
values.  NOTE: There is a slight API change when using the passed in
strings as is.  We now require the passed in strings to no longer have
leading or trailing whitespace.  This isn't a problem as the only callers
have already done this before passing the strings to the affected
function.

ASTERISK-28013 #close

Change-Id: Ia564825a8a95e085fd17e658cb777fe1afa8091a

This adds a module which registers a CLI command that can set the
REMB bitrate value for REMB as it enters or exits Asterisk. This
allows you to ignore what Asterisk or a client produces and is
useful for demonstrations.

This does not generate REMB frames, however, but just modifies
them as they flow to or from a channel.

Change-Id: Ib089427c46a4a36d645cecfe02406adb38c17bec

This change brings in PJSIP 2.8, removes all the patches
that were merged upstream, and makes a minor change to
support a breaking change that was done.

ASTERISK-28059

Change-Id: I5097772b11b0f95c3c1f52df6400158666f0a189

Given a scenario where a session refresh was done with a removed
stream we would always add a removed stream to the outgoing SDP
even if one did not already exist.

This change makes it so that a removed stream is only placed into
the SDP if one already exists.

ASTERISK-28047

Change-Id: Ibb97d21cdeb87a8acae0c720861b0ff255708442

Change-Id: Id2a916ff9448706090e72ff2c7fb3f5ba24a05df

Change-Id: Ib0a645d6985de5757cc4399ed2524b2d02c4f342

Both pjsip_tx_data.tp_info.dst_name and pjsip_rx_data.pkt_info.src_name
store IPv6 addresses without enclosing brackets. This causes some log
output to be confusing because it is difficult to separate the IPv6
address from a port specification.

* Use pj_sockaddr_print() along with pjsip_tx_data.tp_info.dst_addr and
  pjsip_rx_data.pkt_info.src_addr where possible for consistent IPv6
  output.

* When a pj_sockaddr is not available, explicitly wrap IPv6 addresses
  in brackets.

* When assigning pjsip_rx_data.pkt_info.src_name ourselves, make sure
  to also set pjsip_rx_data.pkt_info.src_addr.

Change-Id: I5cfe997ced7883862a12b9c7d8551d76ae02fcf8

As they're not actively used, they only grow stale. The moduleinfo field itself
is kept in Asterisk 13/15 for ABI compatibility.

ASTERISK-28046 #close

Change-Id: I8df66a7007f807840414bb348511a8c14c05a9fc

The bundled version of pjproject has a patch for Solaris compatability
that changes the definition of various socket structures which we need
to account for when compiling against a non-bundled version.

ASTERISK-28049 #close

Change-Id: Ia1ea47c433fc2d915115193ee889a752373925f0

On MOH activation, moh_files_readframe() is called while the current
stream attached to the channel is NULL and it calls ast_moh_files_next()
immediately.  However, it won't call ast_moh_files_next() again if sample
reading fails.  The failure may occur because res_musiconhold retains the
last sample reading position in the channel data and MOH during the
previous hold/retrieve just reached EOF.  Obviously, a bit of bad luck is
required here.

* Restructured moh_files_readframe() to try a second time to start MOH if
there was no stream setup and the saved position was at EOF.  Also added
comments describing what is going on for each step.

ASTERISK-28029

Change-Id: I1508cf2c094f8feca22d6f76deaa9fdfa9944860

Currently, to convert from a pj_sockaddr to an ast_sockaddr, the address
needs to be rendered to a string and then parsed into the correct
structure. This also involves a call to getaddrinfo(3). The same is true
for the inverse operation.

Instead, because we know the internal structure of both ast_sockaddr and
pj_sockaddr, we can translate directly between the two without the
need for an intermediate string.

Change-Id: If0fc4bba9643f755604c6ffbb0d7cc46020bc761

ASTERISK-27988

Change-Id: Iccafdd0552ea8aaed647620fb14499f1bf341843

The T.38 fax gateway state machine can cause the fax gateway
to be started more than once on a channel depending on the
responses of the remote endpoint. This would previously leak
the channel name, channel unique id, and underlying fax engine
state. This change instead makes it so that if the fax gateway
session is already present and not reserved the fax gateway
is not started again.

ASTERISK-27981

Change-Id: I552d95086860cb18f2522ee40ef47b13b6da2e0e

SIP responses over WebSockets when the client is using IPv6 have invalid
Via headers according to RFC 3261. The 'received' header parameter
should not be wrapped in brackets if it is an IPv6 address.

When src_name is populated by the built-in PJSIP transports, the code
uses pj_sockaddr_print() with 'flags' set to 0, meaning that the
brackets are not rendered around IPv6 addresses.

This may be related to ASTERISK~27101.

See also: https://github.com/onsip/SIP.js/pull/594

ASTERISK-28020 #close

Change-Id: I8ea9d289901b837512bee2ca2535e3dc14f04d77

This new option can be passed for ./configure or
./tests/CI/buildAsterisk.sh to prevent download/install of binary
modules.

Normally enabling the categories MENUSELECT_CODECS or MENUSELECT_RES
will result in binary modules being enabled even if the build target is
incompatible with those modules.  This includes CI scripts which enable
categories before disabling specific modules.

If more binary modules are offered in the future this will help avoid
accidentally downloading them if unwanted or incompatible.  Adding a
binary module will only require creating a new menuselect entry similar
to the existing ones, it will not be necessary to modify the CI scripts.

Change-Id: I6b1bd1c75a2e48f05b8b8a45b7a7a2d00a079166

The realtime text timer pops regularly and sends text frames even if
the buffer is empty. This causes a lot of unecessary debug logging.

* Made red_write() test if we need to send a frame before calling
ast_rtp_write()

ASTERISK-28002
Reported by: Emmanuel BUU
Tested by: Emmanuel BUU

Change-Id: Icf81310c3b8080b615a42060afc02ab41f9523dd

When a Contact is updated the only material change that qualify
support cares about is the underlying configuration for the AOR.
In this case we will update things with the new AOR information but
otherwise the callback to indicate the Contact has changed can be
ignored.

This is because it is only when a Contact is added or deleted that
material changes occur within the qualify support. An update can't
change the URI since it would result in a new Contact so it can be
ignored.

Change-Id: I2f97ebfa79969a36a97bb7b9afd5b6268cf1a07d

We were still getting crashes after the first fix.  Somehow we receive a
non-2xx final response before we get a 200 final response.  With the
failure response we had already cleaned up and destroyed some data
structures.  When the unexpected 200 response comes in we crash.

* Add protection code to prevent processing another final T.38 reINVITE
response.

ASTERISK-27944

Change-Id: I8b5baba8d07fe4d63f0d7d05d3eb9a3d27d40a74

The allow_unqualified_fetch option for the sorcery realtime backend
blocked actually fetching all rows when the option is set to warn.

* Made issue a warning and actually do the request when
allow_unqualified_fetch=warn is set.

Change-Id: I74456c80a03a62dce66fc3dc3cb0cf2351ac4312

Compiling without SRTP support installed resulted in some unused variable
warnings.  These warnings also showed that the srtp variable was obtained
and passed around some functions but not really used even when a system
has SRTP installed.

Change-Id: I6daad34be3e89b19adef6e2fbe738018975155fc

If in the initial sdp the caller doesn't include the line
a=rtcp-mux

Then asterisk shoud not include rtcp-mux in the response regardless
of rtcp-mux being enabled on the endpoint

ASTERISK-28007 #close

Change-Id: I58e9b9f40a139afc0da5de41906cc608fb62adc7

Change-Id: I3f396316bb40d1ae6e91f5f688042420f1a540ed

Cleanup idle check scheduled events at shutdown.

Change-Id: I61bfbb56bac69fe840c3242927d31ff3593be461

This makes it possible for REF_DEBUG to report no leaks when loading
res_odbc.

Change-Id: I1a3dea786bd6e7f4820a6dd5cbaa197fa783ce93

sip_options_get_endpoint_state_compositor_state leaked a reference to
the first available endpoint state compositor that was found.

Change-Id: Idb6be19f7219b6eed1dfb19c1e740dd40cb3fdc7

This change adds the "party" parameter to the Remote-Party-ID header
which indicates which party the header information is applicable
to. In Asterisk this is determined on whether we are the calling
or called party. This is added to improve interoperability with some
implementations.

ASTERISK-28006

Change-Id: I1eec3e377ffff8633b5c1dd59a05e9533122cfca

When a conference contained a mixture of audio/video and audio-only
users, a NOTICE message would pop up stating there are no joint
capabilities between streams. This happens because streams can never be
removed, but they can be in a REMOVED state. If we have the scenario
where user A joins with audio/video, user B joins with audio-only, and
user C joins with audio/video, then user A leaves, the message would
be triggered. That removed stream is still in the SDP, but Asterisk
would pass it through, causing it to be seen as a ulaw stream. A check
has been added for removed streams, setting their status to REMOVED when
handling negotiated SDPs.

Also addressed an issue where user A joins, then user B joins but does
not receive video until much later. Full frames were not being sent,
causing some PLI from the browser. Because the video was flowing in one
direction, the browser sets the SSRC to 1, but Asterisk was dropping the
PLI because of that. Added a check to see if the SSRC is 1 or not, which
sends full frames and allows video to flow between user A and user B.
This should only happen when dealing with PSFB or FUR, and in the case
of PSFB, only for PLI.

ASTERISK-27398

Change-Id: I26e7c6f101bc119549eeca406b5bcd25ad8ebc5e

The authors of PJProject undef s_addr because of some issue in Microsoft
Windows. However in Oracle Solaris, s_addr is not a structure member, but
defined to map to the real structure member.

Updates the patch from ASTERISK_20366

ASTERISK-27997

Change-Id: I8223026d4d54e2a46521085fcc94bfa6ebe35b11

* Don't include pjlib.h twice in res_pjsip.h
* Consistently use #include <> form for pjproject includes.
(pjsip.h and pjlib.h)

Change-Id: I3f7b42044840de64edf7e9d7695cb60c45990dc7

If asterisk offer an endpoint with SRTP and that endpoint respond
with non srtp, in that case channel(rtp,secure,audio) reply wrong
status.

Why delete flag AST_SRTP_CRYPTO_OFFER_OK while check identical remote_key:
Currently this flag has being set redundantly. In either case identical
or different remote_key this flag has being set. So if we
don't set it while we receive identical remote_key or non SRTP SDP
response then we can take decision of srtp use by using that flag.

ASTERISK-27999

Change-Id: I29dc2843cf4e5ae2604301cb4ff258f1822dc2d7

This change removes a sorcery lookup for retrieving all
contacts at the end of the registration process by keeping
track of the contacts that are added/updated/deleted.

This ensures at the end of the process the container of
contacts we have is the current state.

Pool usage has also been reduced by allocating one for
usage throughout the handling of a REGISTER and resetting
it to a clean state. This ensures that in most cases
we allocate once and just reuse it.

ASTERISK-28001

Change-Id: I1a78b2d46f9a2045dbbff1a3fd6dba84b612b3cb