This patch fixes several issues reported by the lgtm code analysis tool:

https://lgtm.com/projects/g/asterisk/asterisk

Not all reported issues were addressed in this patch. This patch mostly fixes
confirmed reported errors, potential problematic code points, and a few other
"low hanging" warnings or recommendations found in core supported modules.
These include, but are not limited to the following:

* innapropriate stack allocation in loops
* buffer overflows
* variable declaration "hiding" another variable declaration
* comparisons results that are always the same
* ambiguously signed bit-field members
* missing header guards

Change-Id: Id4a881686605d26c94ab5409bc70fcc21efacc25

ASTERISK-28590 #close

Change-Id: I51abce00c04d0a06550bda5205580705185b9c1c

Serializer pools have previously existed in Asterisk. However, for the most
part the code has been duplicated across modules. This patch abstracts the
code into an 'ast_serializer_pool' object. As well the code is now centralized
in serializer.c/h.

In addition serializer pools can now optionally be monitored by a shutdown
group. This will prevent the pool from being destroyed until all serializers
have completed.

Change-Id: Ib1e906144b90ffd4d5ed9826f0b719ca9c6d2971

Both res_pjsip and res_pjsip_mwi made use of serializer pools. However, they
both implemented their own serializer pool functionality that was pretty much
identical in each of the source files. This patch removes the duplicated code,
and uses the new 'ast_serializer_pool' object instead.

Additionally res_pjsip_mwi enables a shutdown group on the pool since if the
timing was right the module could be unloaded while taskprocessor threads still
needed to execute, thus causing a crash.

Change-Id: I959b0805ad024585bbb6276593118be34fbf6e1d

Add a new dialplan function PJSIP_MOH_PASSTHROUGH that allows
the on-hold behavior to be controlled on a per-call basis

ASTERISK-28542 #close

Change-Id: Iebe905b2ad6dbaa87ab330267147180b05a3c3a8

Previous to this patch passing a NULL tag to ao2_alloc or ao2_ref based
functions would result in the reference not being logged under
REF_DEBUG.  This could sometimes cause inaccurate logging if NULL was
accidentally passed to a reference action.  Now reference logging is
only disabled by option passed to the allocation method.

Change-Id: I3c17d867d901d53f9fcd512bef4d52e342637b54

This change adds support to the JITTERBUFFER dialplan function
for audio and video synchronization. When enabled the RTCP SR
report is used to produce an NTP timestamp for both the audio and
video streams. Using this information the video frames are queued
until their NTP timestamp is equal to or behind the NTP timestamp
of the audio. The audio jitterbuffer acts as the leader deciding
when to shrink/grow the jitterbuffer when adaptive is in use. For
both adaptive and fixed the video buffer follows the size of the
audio jitterbuffer.

ASTERISK-28533

Change-Id: I3fd75160426465e6d46bb2e198c07b9d314a4492

This change adds H.265/HEVC as a known codec and creates a cached
"h265" media format for use.

Note that RFC 7798 section 7.2 also describes additional SDP
parameters. Handling of these is not yet supported.

ASTERISK-28512

Change-Id: I26d262cc4110b4f7e99348a3ddc53bad0d2cd1f2

When modifying an already defined variable in some channel drivers they
add a new variable with the same name to the list, but that value is
never used, only the first one found.

Introduce ast_variable_list_replace() and use it where appropriate.

ASTERISK-23756 #close
Patches:
  setvar-multiplie.patch submitted by Michael Goryainov

Change-Id: Ie1897a96c82b8945e752733612ee963686f32839

Added unit tests for RTCP video stats. These tests include NACK, REMB,
FIR/FUR/PLI, SR/RR/SDES, and packet loss statistics. The REMB and FIR
tests are currently disabled due to a bug. We expect to receive a
compound packet, but the code sends this out as a single packet, which
the browser accepts, but makes Asterisk upset.

While writing these tests, I noticed an issue with NACK as well. Where
it is handling a received NACK request, it was reading in only the first
8 bits of following packets that were also lost. This has been changed
to the correct value of 16 bits.

Also made a minor fix to the data buffer unit test.

Change-Id: I56107c7411003a247589bbb6086d25c54719901b

The new function takes in a pointer to an ast_sockaddr structure,
a hostname and an optional port and then dispatches parallel
"AAAA" and "A" record queries.  If an "AAAA" record is returned,
it's parsed into the ast_sockaddr structure along with the port
if it was supplied.  If no "AAAA" record was returned, the
first "A" record returned (if any) is parsed instead.

This is a synchronous call.  If you need asynchronous lookups,
use ast_dns_query_set_resolve_async and roll your own.

Change-Id: I194b0b0e73da94b35cc35263a868ffac3a8d0a95

There are 4 scenarios to consider when capturing audio from a channel
with an audiohook:

 1. There is no rx and no tx audio, so return nothing.
 2. There is rx but no tx audio, so return rx.
 3. There is tx but no rx audio, so return tx.
 4. There is rx and tx audio, so mix them and return.

The file passed as the primary argument to MixMonitor will be written to
in scenarios 2, 3, and 4. However, if you pass the r() and t() options
to MixMonitor, a frame will only be written to the r() file if there was
rx audio and a frame will only be written to the t() file if there was
tx audio.

If you subsequently take the r() and t() files and try to mix them, the
sides of the conversation will 'drift' and be non-representative of the
user experience.

This patch adds a new 'S' option to MixMonitor that injects a frame of
silence on either the r() side or the t() side of the channel so that
when later mixed, there is no such drift.

Change-Id: Ibf5ed73a811087727bd561a89a59f4447b4ee20e

When updating times on CDR or CEL records using the time at which
it is done can result in times being incorrect if the system is
heavily loaded and stasis message processing is delayed.

This change instead makes it so CDR and CEL use the time at which
the stasis messages that drive the systems are created. This allows
them to be backed up while still producing correct records.

ASTERISK-28498

Change-Id: I6829227e67aefa318efe5e183a94d4a1b4e8500a

Change-Id: I8b8ed97001446fab0c14d7c89391ee572fb29dd6

When fixing ASTERISK~24212, a change was done so a scheduled callback could not
be removed while it was running. The caller of ast_sched_del would have to wait.

However, when the caller of ast_sched_del is the callback itself (however wrong
this might be), this new check would cause a deadlock: it would wait forever
for itself.

This changeset introduces an additional check: if ast_sched_del is called
by the callback itself, it is immediately rejected (along with an ERROR log and
a backtrace). Additionally, the AST_SCHED_DEL_UNREF macro is adjusted so the
after-ast_sched_del-refcall function is only run if ast_sched_del returned
success.

This should fix the following spurious race condition found in chan_sip:
- thread 1: schedule sip_poke_peer_now (using AST_SCHED_REPLACE)
- thread 2: run sip_poke_peer_now
- thread 2: blank out sched-ID (too soon!)
- thread 1: set sched-ID (too late!)
- thread 2: try to delete the currently running sched-ID

After this fix, an ERROR would be logged, but no deadlocks (in do_monitor) nor
excess calls to sip_unref_peer(peer) (causing double frees of rtp_instances and
other madness) should occur.

(Thanks Richard Mudgett for reviewing/improving this "scary" change.)

Note that this change does not fix the observed race condition: unlocked
access to peer->pokeexpire (and potentially other scheduled items in chan_sip),
causing AST_SCHED_DEL_UNREF to look at a changing id. But it will make the
deadlock go away. And in the observed case, it will not have adverse affects
(like memory leaks) because the scheduled item is removed through a different
path.

ASTERISK-28282

Change-Id: Ic26777fa0732725e6ca7010df17af77a012aa856

** Note **

This patch is meant to be the minimum needed in order for the MWI core to use
the now underlying stasis_state module. As such it does not completely remove
its reliance on the stasis_cache. Doing so has allowed current consumers to
not have to change, and update those code paths for this patch. When time
allows, subsequent patches can/will be made to those consumers to take advantage
of some of the new MWI API included here. Thus, eventually and ultimately
removing MWI dependency on the stasis_cache.

** End Note **

This patch makes it so the MWI core now takes advantage of the new stasis_state
API. Consumers of MWI should no longer need to depend upon stasis topic pooling,
and the stasis cache directly. Similar functionality and implementation details
have now been pushed into the stasis_state module. However, all MWI state should
be accessed via the MWI API itself.

As such a few new methods, and constructs have been added to the MWI core that
facilitate consumer publishing, subscribing, and iterating over MWI state data.

* ast_mwi_subscriber *

Created via ast_mwi_add_subscriber, a subscriber subscribes to a given mailbox
in order to receive updates about the given mailbox. Adding a subscriber will
create the underlying topic, and associated state data if those do not already
exist for it. The topic, and last known state data is guaranteed to exist for
the lifetime of the subscriber.

* ast_mwi_publisher *

Before publishing to a particular topic a publisher should be created. This can
be achieved by using ast_mwi_add_publisher. Publishing to a mailbox should then
be done using one of the MWI publish functions. This ensures the message is
published to the appropriate topic, and the last known state is maintained.

* ast_mwi_observer *

Add an observer in order to watch for particular MWI module related events. For
instance if a submodule needs to know when a subscription is added to any
mailbox an observer can be added to watch for that.

* other *

Urgent message count is now part of the published MWI state object. Also state
can be iterated over using defined callbacks.

ASTERISK-28442

Change-Id: I93f935f9090cd5ddff6d4bc80ff90703c05cf776

This new module describes an API that can be thought of as a combination of
stasis topic pools, and caching. Except, hopefully done in a more efficient
and less memory "leaky" manner.

The API defines methods, and data structures for managing, and tracking
published message state through stasis. By adding a subscriber or publisher,
consumers can more easily track the lifetime of the contained state. For
instance, when no more publishers and/or subscribers have need of the topic,
and associated state its data is removed from the managed container.

* stasis_state_manager *

The manager stores and well, manages state data. Each state is an association
of a unique stasis topic, and the last known published stasis message on that
topic. There is only ever one managed state object per topic. For each topic
all messages are forwarded to an "all" topic also maintained by the manager.

* stasis_state_subscriber *

Topic and state can be created, or referenced within the manager by adding a
stasis_state_subscriber. When adding a subscriber if no state currently exists
new managed state is immediately created. If managed state already exists then
a new subscriber is created referencing that state. The managed state is
guaranteed to live throughout the subscriber's lifetime. State is only removed
from the manager when no other entities require it.

* stasis_state_publisher *

Topic and state can be created, or referenced within the manager by also adding
a stasis_state_publisher. When adding a publisher if no state currently exists
new managed state is created. If managed state already exists then a new
publisher is created referencing that state. The managed state is guaranteed to
live throughout the publisher's lifetime. State is only removed from the
manager when no other entities require it.

* stasis_state_observer *

Some modules may wish to watch for, and react to managed state events. By
registering a state observer, and implementing handlers for the desired
callbacks those modules can do so.

* other *

Callbacks also exist that allow consumers to iterate over all, or some of the
managed state.

ASTERISK-28442

Change-Id: I7a4a06685a96e511da9f5bd23f9601642d7bd8e5

When a channel already in a conference bridge is attended transfered
to another extension, or when an existing call is attended
transferred into a conference bridge, we now generate ConfbridgeJoin
and ConfbridgeLeave events for the entering and departing channels.

Change-Id: Id7709cfbceb26fbcb828b2d0d2a6b2fbeaf028e1

This change adds support for larger TLS certificates by allowing
OpenSSL to fragment the DTLS packets according to the configured
MTU. By default this is set to 1200.

This is accomplished by implementing our own BIO method that
supports MTU querying. The configured MTU is returned to OpenSSL
which fragments the packet accordingly. When a packet is to be
sent it is done directly out the RTP instance.

ASTERISK-28018

Change-Id: If2d5032019a28ffd48f43e9e93ed71dbdbf39c06

This patch adds basic Asterisk channel statistics to the res_prometheus
module. This includes:

* asterisk_calls_sum: A running sum of the total number of
  processed calls

* asterisk_calls_count: The current number of calls

* asterisk_channels_count: The current number of channels

* asterisk_channels_state: The state of any particular channel

* asterisk_channels_duration_seconds: How long a channel has existed,
  in seconds

In all cases, enough information is provided with each channel metric
to determine a unique instance of Asterisk that provided the data, as
well as the name, type, unique ID, and - if present - linked ID of each
channel.

ASTERISK-28403

Change-Id: I0db306ec94205d4f58d1e7fbabfe04b185869f59

Prometheus is the defacto monitoring tool for containerized applications.
This patch adds native support to Asterisk for serving up Prometheus
compatible metrics, such that a Prometheus server can scrape an Asterisk
instance in the same fashion as it does other HTTP services.

The core module in this patch provides an API that future work can build
on top of. The API manages metrics in one of two ways:
(1) Registered metrics. In this particular case, the API assumes that
    the metric (either allocated on the stack or on the heap) will have
    its value updated by the module registering it at will, and not
    just when Prometheus scrapes Asterisk. When a scrape does occur,
    the metrics are locked so that the current value can be retrieved.
(2) Scrape callbacks. In this case, the API allows consumers to be
    called via a callback function when a Prometheus initiated scrape
    occurs. The consumers of the API are responsible for populating
    the response to Prometheus themselves, typically using stack
    allocated metrics that are then formatted properly into strings
    via this module's convenience functions.

These two mechanisms balance the different ways in which information is
generated within Asterisk: some information is generated in a fashion
that makes it appropriate to update the relevant metrics immediately;
some information is better to defer until a Prometheus server asks for
it.

Note that some care has been taken in how metrics are defined to
minimize the impact on performance. Prometheus's metric definition
and its support for nesting metrics based on labels - which are
effectively key/value pairs - can make storage and managing of metrics
somewhat tricky. While a naive approach, where we allow for any number
of labels and perform a lot of heap allocations to manage the information,
would absolutely have worked, this patch instead opts to try to place
as much information in length limited arrays, stack allocations, and
vectors to minimize the performance impacts of scrapes. The author of
this patch has worked on enough systems that were driven to their knees
by poor monitoring implementations to be a bit cautious.

Additionally, this patch only adds support for gauges and counters.
Additional work to add summaries, histograms, and other Prometheus
metric types may add value in the future. This would be of particular
interest if someone wanted to track SIP response types.

Finally, this patch includes unit tests for the core APIs.

ASTERISK-28403

Change-Id: I891433a272c92fd11c705a2c36d65479a415ec42

Added a conversion for umax (largest maximum sized integer allowed). Adjusted
the other current conversion functions (uint and ulong) to be derivatives of
the umax conversion since they are simply subsets of umax.

Also made the negative check move the pointer on spaces since strtoumax does it
anyways.

Change-Id: I56c2ef2629d49b524c8df58af12951c181f81f08

After a bridge has been deleted the stasis control will depart
the channel and might attempt to re-add it to the dial bridge.

The later can fail and this can lead to a situation that the stasis
control is unlinked but the after_bridge_cb_failed cb is executed trying
to access a dangling control object.

Fix it by calling the after_cb's before bridge_channel_impart_signal.

ASTERISK-26718

Change-Id: Ib4e8f70d7a21bd54afe3cb51cc6717ef7c355496

When producing a combined REMB value the normal behavior
is to have a REMB value which is unique for each sender
based on all of their receivers. This can result in one
sender having low bitrate while all the rest are high.

This change adds "all" variants which produces a bridge
level REMB value instead. All REMB reports are combined
together into a single REMB value that is the same for
each sender.

ASTERISK-28401

Change-Id: I883e6cc26003b497c8180b346111c79a131ba88c

The transport-cc draft is a mechanism by which additional information
about packet reception can be provided to the sender of packets so
they can do sender side bandwidth estimation. This is accomplished
by having a transport specific sequence number and an RTCP feedback
message. This change implements this in the receiver direction.

For each received RTP packet where transport-cc is negotiated we store
the time at which the RTP packet was received and its sequence number.
At a 1 second interval we go through all packets in that period of time
and use the stored time of each in comparison to its preceding packet to
calculate its delta. This delta information is placed in the RTCP
feedback message, along with indicators for any packets which were not
received.

The browser then uses this information to better estimate available
bandwidth and adjust accordingly. This may result in it lowering the
available send bandwidth or adjusting how "bursty" it can be.

ASTERISK-28400

Change-Id: I654a2cff5bd5554ab94457a14f70adb71f574afc

Added RINGTIME, RINGTIME_MS, PROGRESSTIME, PROGRESSTIME_MS variables filled
at the earliest received PROGRESS or RINGING.
Added millisecond versions of DIALEDTIME and ANSWEREDTIME.

Added millisecond versions of ast_channel_get_up_time and
ast_channel_get_duration in channel.c.

ASTERISK-28363

Change-Id: If95f1a7d8c4acbac740037de0c6e3109ff6620b1

There is enough MWI functionality to warrant it having its own 'c' and header
files. This patch moves all current core MWI data structures, and functions
into the following files:

main/mwi.h
main/mwi.c

Note, code was simply moved, and not modified. However, this patch is also in
preparation for core MWI changes, and additions to come.

Change-Id: I9dde8bfae1e7ec254fa63166e090f77e4d3097e0

Added a new PJSIP global setting called norefersub.
Default is true to keep support working as before.

res_pjsip_refer:  Configures PJSIP norefersub capability accordingly.

Checks the PJSIP global setting value.
If it is true (default) it adds the norefersub capability to PJSIP.
If it is false (disabled) it does not add the norefersub capability
to PJSIP.

This is useful for Cisco switches that do not follow RFC4488.

ASTERISK-28375 #close
Reported-by: Dan Cropp

Change-Id: I0b1c28ebc905d881f4a16e752715487a688b30e9

Asterisk assumes that dlopen() will always run the constructor of a
shared library and every dlclose() will run its destructor. But dlopen()
may be permanent, meaning the constructor will only be run once, as is
the case with musl libc.

With a permanent dlopen() the Asterisk module loader does not work
correctly, because it's expectations regarding when the constructors and
destructors are run are not met. In fact a segmentation fault will occur
when the first module is "re-opened" that has AST_MODFLAG_GLOBAL_SYMBOLS
set (the dlopen() does not call the constructor, resource_being_loaded
is not set to NULL, then strlen is called with NULL instead of a string,
see issue ASTERISK-28319).

This commit adds code to the loader that will manually run the
constructors/destructors of the (non-builtin) modules where needed. To
achieve this a new ao2 container (linked list) is started and filled
with objects that contain the names of the modules and the pointers to
their respective info structs.

This behavior can be activated when configuring Asterisk
(--enable-permanent-dlopen). By default this is disabled, of course.

ASTERISK-28319 #close

Signed-off-by: Sebastian Kemper <sebastian_ml@gmx.net>
Change-Id: I86693a0ecf25d5ba81c73773a03df4abc3426875

Added topic_all container for centralizing the topic. This makes more
easier to managing the topics.

Added cli commands.
stasis show topics : It shows all registered topics.
stasis show topic <name> : It shows speicifed topic's detail info.

ASTERISK-28264

Change-Id: Ie86d125d2966f93de74ee00f47ae6fbc8c081c5f

It was difficult to check the channel's current application and
parameters using ARI for current channels. Added app_name, app_data
items to show the current application information.

ASTERISK-28343

Change-Id: Ia48972b3850e5099deab0faeaaf51223a1f2f38c

Since the new names went in, the maximum taskprocessor name is too
short.  This patch increases the name field to a length to better
handle the new names.

Change-Id: I32f32d6926f25c8ef5a91303fd2988d2c2858877

Added ability to specifiy a wizard is read-only when applying
it to a specific object type.  This allows you to specify
create, update and delete callbacks for the wizard but limit
which object types can use them.

Added the ability to allow an object type to have multiple
wizards of the same type.  This is indicated when a wizard
is added to a specific object type.

Added 3 new sorcery wizard functions:

* ast_sorcery_object_type_insert_wizard which does the same thing
  as the existing ast_sorcery_insert_wizard_mapping function but
  accepts the new read-only and allot-duplicates flags and also
  returns the ast_sorcery_wizard structure used and it's internal
  data structure. This allows immediate use of the wizard's
  callbacks without having to register a "wizard mapped" observer.

* ast_sorcery_object_type_apply_wizard which does the same
  thing as the existing ast_sorcery_apply_wizard_mapping function
  but has the added capabilities of
  ast_sorcery_object_type_insert_wizard.

* ast_sorcery_object_type_remove_wizard which removes a wizard
  matching both its name and its original argument string.

* The original logic in __ast_sorcery_insert_wizard_mapping was moved
  to __ast_sorcery_object_type_insert_wizard and enhanced for the
  new capabilities, then __ast_sorcery_insert_wizard_mapping was
  refactored to just call __ast_sorcery_insert_wizard_mapping.

* Added a unit test to test_sorcery.c to test the read-only
  capability.

Change-Id: I40f35840252e4313d99e11dbd80e270a3aa10605

This might be useful in situations where you are loading an undetermined number
of items into a vector and don't want to keep (potentially) 2x the necessary
memory around indefinitely.

Change-Id: I9711daa0fe01783fc6f04c5710eba84f2676d7b9

Added ARI resource for channel statistics.
GET /ari/channels/{channelId}/rtp_statistics : It returns given
channel's rtp statistics detail.

ASTERISK-28320

Change-Id: I4343eec070438cec13f2a4f22e7fd9e574381376

This reverts commit 1c8378bb.

Change-Id: I1b9227b263c3dc4246a50aebf52a7640a0f7ea07

Topic names now follow: <subsystem>:<functionality>[/<object>]

This ensures that they are all unique, and also provides better
insight in to what each topic is for.

Subscriber ids now also use the main topic name they are
subscribed to and an incrementing integer as their identifier to
make it easier to understand what the subscription is primarily
responsible for.

Both the CLI commands for listing topic and subscription statistics
now sort to make it a bit easier to see what is going on.

Subscriptions will now show all topics that they are receiving messages
from, not just the main topic they were subscribed to.

ASTERISK-28335

Change-Id: I484e971a38c3640f2bd156282e532eed84bf220d

chan_sip will always ignore 183 responses that do not contain SDP
however, chan_pjsip will currently always translate it into a
183 with SDP.  This new flag allows chan_pjsip to have the same
behavior as chan_sip.

ASTERISK-28322 #close

Change-Id: If81cfaa17c11b6ac703e3d71696f259d86c6be4a

Add a json_pack at startup that will fail if runtime links against a
library older than jansson-2.11.

Change-Id: I101aebafe0f9407650206f7c552dad3d69377b5a

strtok() uses a static buffer, making it not thread safe.

Also add a #define to cause a compile failure if strtok is used.

Change-Id: Icce265153e1e65adafa8849334438ab6d190e541