There are various places in Asterisk - specifically in regards to
database integration - where having some kind of UTF-8 validation would
be beneficial. This patch adds:

* Functions to validate that a given string contains only valid UTF-8
  sequences.

* A function to copy a string (similar to ast_copy_string) stopping when
  an invalid UTF-8 sequence is encountered.

* A UTF-8 validator that allows for progressive validation.

All of this is based on the excellent UTF-8 decoder by Björn Höhrmann.
More information is available here:

    https://bjoern.hoehrmann.de/utf-8/decoder/dfa/

The API was written in such a way that should allow us to replace the
implementation later should we determine that we need something more
comprehensive.

Change-Id: I3555d787a79e7c780a7800cd26e0b5056368abf9

Currently, if the bridge has created by the ARI, the video_mode
parameter was
not shown in the BridgeCreated event correctly.

Fixed it and added video_mode shown in the 'bridge show <bridge id>'
cli.

ASTERISK-28987

Change-Id: I8c205126724e34c2bdab9380f523eb62478e4295

If an ACL is misconfigured in the realtime database (for instance, the
"rule" is blank) and Asterisk attempts to read the ACL, Asterisk will
crash.

ASTERISK-28978 #close

Change-Id: Ic1536c4df856231bfd2da00128f7822224d77610

Prior to making any modifications to the pjsip infrastructure
for ACN, I've added the tracing functions to the existing code.
This should make the final commit easier to review, but we can also
now run a "before and after" trace.

No functional changes were made with this commit.

Change-Id: Ia83a1a2687ccb96f2bc8a2a3928a5214c4be775c

 * ast_frame_subclass2str() and ast_frame_type2str() now return
   a pointer to the buffer that was passed in instead of void.
   This makes it easier to use these functions inline in
   printf-style debugging statements.

 * Added many missing control frame entries in
   ast_frame_subclass2str.

Change-Id: Ifd0d6578e758cd644c96d17a5383ff2128c572fc

Tracing through synchronous tasks was a little troublesome because
the new thread's stack counter reset to 0.  This change allows
a synchronous task to set its trace level to be the same as the
thread that pushed the task.  For now, the task's level has to be
passed in the task's data structure but a future enhancement to the
taskprocessor subsystem could automatically set the trace level
of the servant to be that of the caller.

This doesn't really make sense for async tasks because you never
know when they're going to run anyway.

Change-Id: Ib8049c0b815063a45d8c7b0cb4e30b7b87b1d825

This patch allows a user of AMI to now specify the type of message
content contained within by setting the 'Content-Type' parameter.

Note, the AMI version has been bumped for this change.

ASTERISK-28945 #close

Change-Id: Ibb5315702532c6b954e1498beddc8855fabdf4bb

The Streams API becomes the home for the core ACN capabilities.
These include...

 * Parsing and formatting of codec negotation preferences.
 * Resolving pending streams and topologies with those configured
   using configured preferences.
 * Utility functions for creating string representations of
   streams, topologies, and negotiation preferences.

For codec negotiation preferences:
 * Added ast_stream_codec_prefs_parse() which takes a string
   representation of codec negotiation preferences, which
   may come from a pjsip endpoint for example, and populates
   a ast_stream_codec_negotiation_prefs structure.
 * Added ast_stream_codec_prefs_to_str() which does the reverse.
 * Added many functions to parse individual parameter name
   and value strings to their respectrive enum values, and the
   reverse.

For streams:
 * Added ast_stream_create_resolved() which takes a "live" stream
   and resolves it with a configured stream and the negotiation
   preferences to create a new stream.
 * Added ast_stream_to_str() which create a string representation
   of a stream suitable for debug or display purposes.

For topology:
 * Added ast_stream_topology_create_resolved() which takes a "live"
   topology and resolves it, stream by stream, with a configured
   topology stream and the negotiation preferences to create a new
   topology.
 * Added ast_stream_topology_to_str() which create a string
   representation of a topology suitable for debug or display
   purposes.
 * Renamed ast_format_caps_from_topology() to
   ast_stream_topology_get_formats() to be more consistent with
   the existing ast_stream_get_formats().

Additional changes:
 * A new function ast_format_cap_append_names() appends the results
   to the ast_str buffer instead of replacing buffer contents.

Change-Id: I2df77dedd0c72c52deb6e329effe057a8e06cd56

Integrated STIR/SHAKEN support with outgoing INVITEs. When an INVITE is
sent, the caller ID will be checked to see if there is a certificate
that corresponds to it. If so, that information will be retrieved and an
Identity header will be added to the SIP message. The format is:

header.payload.signature;info=<public_key_url>alg=ES256;ppt=shaken

Header, payload, and signature are all BASE64 encoded. The public key
URL is retrieved from the certificate. Currently the algorithm and ppt
are ES256 and shaken, respectively. This message is signed and can be
used for verification on the receiving end.

Two new configuration options have been added to the certificate object:
attestation and origid. The attestation is required and must be A, B, or
C. origid is the origination identifier.

A new utility function has been added as well that takes a string,
allocates space, BASE64 encodes it, then returns it, eliminating the
need to calculate the size yourself.

Change-Id: I1f84d6a5839cb2ed152ef4255b380cfc2de662b4

When requesting a Local channel the requested stream topology
or a converted stream topology will now be placed onto the
resulting channels.

Frames written in on streams will now also preserve the stream
identifier as they are queued on the opposite channel.

Finally when a stream topology change is requested it is
immediately accepted and reflected on both channels. Each
channel also receives a queued frame to indicate that the
topology has changed.

ASTERISK-28938

Change-Id: I4e9d94da5230d4bd046dc755651493fce1d87186

This patch fixes a few compile warnings/errors that now occur when using gcc
10+.

Also, the Makefile.rules check to turn off partial inlining in gcc versions
greater or equal to 8.2.1 had a bug where it only it only checked against
versions with at least 3 numbers (ex: 8.2.1 vs 10). This patch now ensures
any version above the specified version is correctly compared.

Change-Id: I54718496eb0c3ce5bd6d427cd279a29e8d2825f9

Added default variable value to fix a compiler error.

Change-Id: I7b592adbb1274dc5464dea1c5e5de0685c928553

Integrated STIR/SHAKEN support with incoming INVITES. Upon receiving an
INVITE, the Identity header is retrieved, parsing the message to verify
the signature. If any of the parsing fails,
AST_STIR_SHAKEN_VERIFY_NOT_PRESENT will be added to the channel for this
caller ID. If verification itself fails,
AST_STIR_SHAKEN_VERIFY_SIGNATURE_FAILED will be added. If anything in
the payload does not line up with the SIP signaling,
AST_STIR_SHAKEN_VERIFY_MISMATCH will be added. If all of the above steps
pass, then AST_STIR_SHAKEN_VERIFY_PASSED will be added, completing the
verification process.

A new config option has been added to the general section for
stir_shaken.conf. "signature_timeout" is the amount of time a signature
will be considered valid. If an INVITE is received and the amount of
time between when it was received and when it was signed is greater than
signature_timeout, verification will fail.

Some changes were also made to signing and verification. There was an
error where the whole JSON string was being signed rather than the
header combined with the payload. This has been changed to sign the
correct thing. Verification has been changed to do this as well, and the
unit tests have been updated to reflect these changes.

A couple of utility functions have also been added. One decodes a BASE64
string and returns the decoded string, doing all the length calculations
for you. The other retrieves a string value from a header in a rdata
object.

Change-Id: I855f857be3d1c63b64812ac35d9ce0534085b913

If a frame is written to a channel in a bridge we
would normally queue this frame up and the channel
thread would then act upon it. If this frame had no
stream mapping on the channel it would then be
discarded.

This change adds a check before the queueing occurs
to determine if a mapping exists. If it does not
exist then the frame is not even queued at all. This
stops a frame duplication from happening and from
the channel thread having to wake up and deal with
it.

Change-Id: I17189b9b1dec45fc7e4490e8081d444a25a00bda

If the bridge show all command could not get the bridge snapshot, it causes null pointer exception.
Fixed it to check the snapshot is null.

ASTERISK-28920

Change-Id: I3521fc1b832bfc69644d0833f2c78177e1e51f58

What's wrong with ast_debug?

  ast_debug is fine for general purpose debug output but it's not
  really geared for scope tracing since it doesn't present its
  output in a way that makes capturing and analyzing flow through
  Asterisk easy.

How is scope tracing better?

  Scope tracing uses the same "cleanup" attribute that RAII_VAR
  uses to print messages to a separate "trace" log level.  Even
  better, the messages are indented and unindented based on a
  thread-local call depth counter.  When output to a separate log
  file, the output is uncluttered and easy to follow.

  Here's an example of the output. The leading timestamps and
  thread ids are removed and the output cut off at 68 columns for
  commit message restrictions but you get the idea.

--> res_pjsip_session.c:3680 handle_incoming PJSIP/1173-00000001
	--> res_pjsip_session.c:3661 handle_incoming_response PJSIP/1173
		--> res_pjsip_session.c:3669 handle_incoming_response PJSIP/
			--> chan_pjsip.c:3265 chan_pjsip_incoming_response_after
				--> chan_pjsip.c:3194 chan_pjsip_incoming_response P
					    chan_pjsip.c:3245 chan_pjsip_incoming_respon
				<-- chan_pjsip.c:3194 chan_pjsip_incoming_response P
			<-- chan_pjsip.c:3265 chan_pjsip_incoming_response_after
		<-- res_pjsip_session.c:3669 handle_incoming_response PJSIP/
	<-- res_pjsip_session.c:3661 handle_incoming_response PJSIP/1173
<-- res_pjsip_session.c:3680 handle_incoming PJSIP/1173-00000001

  The messages with the "-->" or "<--" were produced by including
  the following at the top of each function:

  SCOPE_TRACE(1, "%s\n", ast_sip_session_get_name(session));

  Scope isn't limited to functions any more than RAII_VAR is.  You
  can also see entry and exit from "if", "for", "while", etc blocks.

  There is also an ast_trace() macro that doesn't track entry or
  exit but simply outputs a message to the trace log using the
  current indent level.  The deepest message in the sample
  (chan_pjsip.c:3245) was used to indicate which "case" in a
  "select" was executed.

How do you use it?

  More documentation is available in logger.h but here's an overview:

  * Configure with --enable-dev-mode.  Like debug, scope tracing
    is #ifdef'd out if devmode isn't enabled.

  * Add a SCOPE_TRACE() call to the top of your function.

  * Set a logger channel in logger.conf to output the "trace" level.

  * Use the CLI (or cli.conf) to set a trace level similar to setting
    debug level... CLI> core set trace 2 res_pjsip.so

Summary Of Changes:

  * Added LOG_TRACE logger level.  Actually it occupies the slot
    formerly occupied by the now defunct "event" level.

  * Added core asterisk option "trace" similar to debug.  Includes
	ability to specify global trace level in asterisk.conf and CLI
	commands to turn on/off and set levels.  Levels can be set
	globally (probably not a good idea), or by module/source file.

  * Updated sample asterisk.conf and logger.conf.  Tracing is
    disabled by default in both.

  * Added __ast_trace() to logger.c which keeps track of the indent
    level using TLS. It's #ifdef'd out if devmode isn't enabled.

  * Added ast_trace() and SCOPE_TRACE() macros to logger.h.
    These are all #ifdef'd out if devmode isn't enabled.

Why not use gcc's -finstrument-functions capability?

  gcc's facility doesn't allow access to local data and doesn't
  operate on non-function scopes.

Known Issues:

  The only know issue is that we currently don't know the line
  number where the scope exited.  It's reported as the same place
  the scope was entered.  There's probably a way to get around it
  but it might involve looking at the stack and doing an 'addr2line'
  to get the line number.  Kind of like ast_backtrace() does.
  Not sure if it's worth it.

Change-Id: Ic5ebb859883f9c10a08c5630802de33500cad027

ASTERISK-28797

Change-Id: Iab364a2c2519fd9d11d1c28293fda43d61b64c28

fork before exec

Posix does only allow async-signal-safe syscalls after fork before exec.
As asterisk ignores this, functions like TrySystem or System sometimes
end up in a deadlocked child process. The patch prevents the use of
non-async-signal-safe syscalls.

ASTERISK-28776

Change-Id: Idc76365c0592ee3f3b3bd72a4f48f7a098978e8e

ast_stream_topology_create_from_format_cap() was setting the
stream->formats directly but not freeing the default formats.  This
causes a memory leak.

* ast_stream_topology_create_from_format_cap() now calls
  ast_stream_set_formats() which properly cleans up the existing
  stream formats.

When cloning a stream, the source stream's format caps _pointer_ is
copied to the new stream and it's reference count bumped.  If
either stream is set to "removed", this will cause _both_ streams
to have their format caps cleared.

* ast_stream_clone() now creates a new format caps object and copies
  the formats from the source stream instead of just copying the
  pointer.

ASTERISK-28870

Change-Id: If697d81c3658eb7baeea6dab413b13423938fb53

Add a new "masquarade" channel event, and use it in app_queue to track unique id's.

Testcase is submitted as https://gerrit.asterisk.org/c/testsuite/+/14210

ASTERISK-28829 #close
ASTERISK-25844 #close

Change-Id: Ifc5f9f9fd70903f3c6e49738d3bc632b085d2df6

These are not provided by standards, and as a result causes failure to
compile on musl.

https://wiki.musl-libc.org/faq.html#Q:-When-compiling-something-against-musl,-I-get-error-messages-about-%3Ccode%3Esys/cdefs.h%3C/code%3E

Change-Id: I6a357cefd106c72cfecafd898638f6b5692c2e05

In a Dialplan, the channel drivers 'chan_sip' and 'chan_iax2' support
the channel items 'secure_bridge_media' and 'secure_bridge_signaling'.
That way, a channel can be forced to use encryption even if not
specified in its configuration.

However, when the Local Proxy kicks in, for example, in case of a
forwarding (SIP status 302), Local Proxy stated it does not know those
items. Consequently, such a call could not be proxied how clever your
Dialplan was. Because local calls within Asterisk are always secure,
Local Proxy accepts such a request now.

ASTERISK-22920

Change-Id: I4c143bb70f686790953cc04c5a4b810bbb03636c

While testing the latest RC on FreeBSD I noticed this new file fails to build. On FreeBSD inlcuding resolv.h requires sockaddr_in to be defined, and it's defined in netinet/in.h. So I added this include.

ASTERISK-28853 #close

Change-Id: I6997daf3956e6eb70ab6cb358628d162fad80079

Some places in Asterisk did not treat the formats on a stream
as immutable when they are.

The ast_stream_get_formats function is now const to enforce this
and parts of Asterisk have been updated to take this into account.
Some violations of this were also fixed along the way.

An additional minor tweak is that streams are now allocated with
an empty format capabilities structure removing the need in various
places to check that one is present on the stream.

ASTERISK-28846

Change-Id: I32f29715330db4ff48edd6f1f359090458a9bfbe

When in a conference bridge it may be necessary to have
text messages disabled for specific participants or for
all. This change adds a configuration option, "text_messaging",
which can be used to enable or disable this on the
user profile. By default existing behavior is preserved
as it defaults to "yes".

ASTERISK-28841

Change-Id: I30b5d9ae6f4803881d1ed9300590d405e392bc13

When the ast_data_buffer_put rejects to add a packet, for example because
the buffer already contains a packet with the same sequence number, the
payload will never be freed, resulting in a memory leak.

The data buffer will now return an error if this situation occurs
allowing the caller to free the payload. The res_rtp_asterisk module
has also been updated to do this.

ASTERISK-28826

Change-Id: Ie6c49495d1c921d5f997651c7d0f79646f095cf1

Allow voice volume to be multiplied or divided by a floating point number.

ASTERISK-28813

Change-Id: I5b42b890ec4e1f6b0b3400cb44ff16522b021c8c

My tester missed this one previously, have confirmed a positive build
this time round.

Change-Id: Id06853375954a200f03f9a1b9c97fe0b10d31fbf

Based on this new endpoint setting, a joint list of preferred codecs
between those received from the Asterisk core (remote), and those
specified in the endpoint's "allow" parameter (local) is created and
is used to create the outgoing SDP offer.

* Add outgoing_call_offer_pref to pjsip_configuration (endpoint)

* Add "call_direction" to res_pjsip_session.

* Update pjsip_session_caps.c to make the functions more generic
  so they could be used for both incoming and outgoing.

* Update ast_sip_session_create_outgoing to create the
  pending_media_state->topology with the results of
  ast_sip_session_create_joint_call_stream().

* The endpoint "preferred_codec_only" option now automatically sets
  AST_SIP_CALL_CODEC_PREF_FIRST in incoming_call_offer_pref.

* A helper function ast_stream_get_format_count() was added to
  streams to return the current count of formats.

ASTERISK-28777

Change-Id: Id4ec0b4a906c2ae5885bf947f101c59059935437

If a frame handling routine returns a list of frames (vs. a single frame)
those frames are never passed to a tech's write_stream handler even if one is
available. For instance, if a codec translation occurred and that codec
returned multiple frames then those particular frames were always only sent
to the tech's "write" handler. If that tech (pjsip for example) was stream
capable then those frames were essentially ignored. Thus resulting in bad
audio.

This patch makes it so the "write_stream" handler is appropriately called
for all cases, and for all frames if available.

ASTERISK-28795 #close

Change-Id: I868faea0b73a07ed5a32c2b05bb9cf4b586f739d

The dial application had 80 characters of destination length
limitation. But this limitation causes unexpected dial string
cut if the dial string is long.

Removed unnecessary limited buffer to support longer dial
destination.

ASTERISK-27946

Change-Id: I72c8f0319a4b47e8180817a66a7e9bde063cb330

named_acl.c (which is really a named_ha) now uses ast_ha_output.

I've also updated main/manager.c to output the actual ACL on "manager
show user <username>" if one is set.  If this works then we can add
similar to other modules as required.

Change-Id: I0ec9876a90dddd379c80ec078d48e3ee6991eb0f

binutils 2.34 merged this commit:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;\
	h=fd3619828e94a24a92cddec42cbc0ab33352eeb4

Which effectively does things like:

-#define bfd_section_size(bfd, ptr) ((ptr)->size)
-#define bfd_get_section_size(ptr) ((ptr)->size)

+#define bfd_section_size(sec) ((sec)->size)

So in order to remain backwards compatible we need to detect this API
change, and adjust accordingly.  The simplest is to notice that the
bfd_get_section_size and bfd_get_section_vma MACROs are no longer
defined, and define then onto the new API.  The alternative is to litter
the code with a number of #ifdef #else #endif splatters right through
the code.

Change-Id: I3efe0f8e8f3e338d16fcbc2b26a505367b6e172f

Change-Id: Ie0eca23b8e6f4c7d9846b6013d79099314d90ef5

Given a scenario where MixMonitor was initiated over AMI it
was possible for the channel and MixMonitor thread to remain
alive past hang up of the channel. This scenario required
the AMI initiated MixMonitor to retrieve the channel, a
hangup to occur on the channel in another thread, and then
for MixMonitor to actually start. If this occurred the
MixMonitor thread would remain alive indefinitely and
the channel reference would remain.

This change ensures that audiohooks are never able to
be attached to channels that have been hung up. An
additional fix has also been done in app_mixmonitor to
properly release the channel reference if this occurs.

ASTERISK-28780

Change-Id: I8044c06daa06f0f16607788c596f55623be26f58

A regular expression in a NAPTR response record can have a trailing
'i' flag to indicate that the expression should be evaluated in a
case-insensitive way. We were not checking for that flag which caused
the record parsing to fail on otherwise valid input.

Although this change will initially go into Asterisk 13, 16, and 17,
it is my intention to replace the majority of this code in 16 and up -
including this fix - by changing enum.c to consume the new DNS API
which duplicates most of this logic already. Asterisk 13 doesn't have
the DNS API, so this fix will be as good as it gets.

ASTERISK-26711 #close
Reported by: Vitold

Change-Id: I33943a5b3e7539c6dca3a5079982ee15a08186f0

The ast_get_txt() API function (and by extension, the TXTCIDNAME
dialplan function) were broken in
65b83815 such that we would never
actually make a DNS TXT query as described.

This patch restores the documented behavior.

ASTERISK-19460 #close
Reported by: George Joseph

Change-Id: I1b19aea711488cb1ecd63843cddce05010e39376

When a text message was received any associated variable was not written to
the ARI TextMessageReceived event. This occurred because Asterisk only wrote
out "send" variables. However, even those "send" variables would fail ARI
validation due to a TextMessageVariable formatting bug.

Since it seems the TextMessageReceived event has never been able to include
actual variables it was decided to remove the TextMessageVariable object type
from ARI, and simply return a JSON object of key/value pairs for variables.
This aligns more with how the ARI sendMessage handles variables, and other
places in ARI.

ASTERISK-28755 #close

Change-Id: Ia6051c01a53b30cf7edef84c27df4ed4479b8b6f

There were a couple places where the format cap function parameter was not
'const' when it should have been. This patch makes them 'const'.

Change-Id: Ife753fb16a962d842a6b44f45363a61a66bfdb2e

There are exceptions for plural objects, but they are detected using the
supplied NUMBER, not using an extra option.

Change-Id: I95d1d1b2796b1aba92048a2dbae8a3856ed8a113