This change adds unit tests for the various API calls relating
to stream topologies. This includes creation, destruction,
inspection, and manipulation.

Through this a few bugs were uncovered in the implementation:

1. Creating a topology using a format capabilities would fail as
the code considered a return value of 0 from the append stream
function to indicate an error which is incorrect.

2. Not all functions which placed a stream into a topology
set the position on the stream itself.

3. Appending a stream would cause a frack if the position
provided was the last one. This occurred because the existing
stream was queried but the index was outside of what the
vector was currently at for size.

ASTERISK-26786

Change-Id: Id5590e87c8a605deea1a89e53169a9c011d66fa0

This change adds the media stream topology definition and API for
accessing and using it.

Some refactoring of the stream was also done.

ASTERISK-26786

Change-Id: Ic930232d24d5ad66dcabc14e9b359e0ff8e7f568

This change adds the media stream definition and API for
accessing and using it. Unit tests have also been written
which exercise aspects of the API.

ASTERISK-26773

Change-Id: I3dbe54065b55aaa51f467e1a3bafd67fb48cac87

When performing an SRV lookup using the ast_srv_lookup function it
did not properly handle the situation where 0 records are returned.
If this happened it would wrongly assume that at least one record
was present.

This change fixes the code so it will exit early if an error occurs
or if 0 records are returned.

ASTERISK-26772
patches:
  srv_lookup.patch submitted by nappsoft (license 6822)

Change-Id: I09b19081c74e0ad11c12bf54a257243b1bcb2351

In ari.conf, when setting the option channelvars, every Stasis channel
snapshot would create a list of variable/value that would not be freed
when the snapshot is freed, resulting in a often-recurring memory
leak.

ASTERISK-26767 #close

Change-Id: Ia37dd9d68063d7f879193df02ede293e5ded716d

Using the timerfd timing module can cause channel freezing, lingering, or
deadlock issues.  The problem is because this is the only timing module
that uses an associated alert-pipe.  When the alert-pipe becomes
unbalanced with respect to the number of frames in the read queue bad
things can happen.  If the alert-pipe has fewer alerts queued than the
read queue then nothing might wake up the thread to handle received frames
from the channel driver.  For local channels this is the only way to wake
up the thread to handle received frames.  Being unbalanced in the other
direction is less of an issue as it will cause unnecessary reads into the
channel driver.

ASTERISK-26716 is an example of this deadlock which was indirectly fixed
by the change that found the need for this patch.

* In channel.c:__ast_queue_frame(): Adding frame lists to the read queue
did not add the same number of alerts to the alert-pipe.  Correspondingly,
when there is an exceptionally long queue event, any removed frames did
not also remove the corresponding number of alerts from the alert-pipe.

ASTERISK-26632 #close

Change-Id: Ia98137c5bf6e9d6d202ce0eb36441851875863f6

A dialplan intercept routine is equivalent to an interrupt routine.  As
such, the routine must be done quickly and you do not have access to the
media stream.  These restrictions are necessary because the media stream
is the responsibility of some other code and interfering with or delaying
that processing is bad.  A possible future dialplan processing
architecture change may allow the interception routine to run in a
different thread from the main thread handling the media and remove the
execution time restriction.

* Made res_agi.c:run_agi() running an AGI in an interception routine run
in DeadAGI mode.  No touchy channel frames.

ASTERISK-25951

ASTERISK-26343

ASTERISK-26716

Change-Id: I638f147ca7a7f2590d7194a8ef4090eb191e4e43

There are several issues with deferring frames that are caused by the
refactoring.

1) The code deferring frames mishandles adding a deferred frame to the
deferred queue.  As a result the deferred queue can only be one frame
long.

2) Deferrable frames can come directly from the channel driver as well as
the read queue.  These frames need to be added to the deferred queue.

3) Whoever is deferring frames is really only doing the __ast_read() to
collect deferred frames and doesn't care about the returned frames except
to detect a hangup event.  When frame deferral is completed we must make
the normal frame processing see the hangup as a frame anyway.  As such,
there is no need to have varying hangup frame deferral methods.  We also
need to be aware of the AST_SOFTHANGUP_ASYNCGOTO hangup that isn't real.
That fake hangup is to cause the PBX thread to break out of loops to go
execute a new dialplan location.

4) To properly deal with deferrable frames from the channel driver as
pointed out by (2) above, means that it is possible to process a dialplan
interception routine while frames are deferred because of the
AST_CONTROL_READ_ACTION control frame.  Deferring frames is not
implemented as a re-entrant operation so you could have the unsupported
case of two sections of code thinking they have control of the media
stream.

A worse problem is because of the bad implementation of the AMI PlayDTMF
action.  It can cause two threads to be deferring frames on the same
channel at the same time.  (ASTERISK_25940)

* Rather than fix all these problems simply revert the API refactoring as
there is going to be only autoservice and safe_sleep deferring frames
anyway.

ASTERISK-26343

ASTERISK-26716 #close

Change-Id: I45069c779aa3a35b6c863f65245a6df2c7865496

If an audiohook is placed on a channel that does not require transcoding,
muting that hook will cause the underlying frames to be muted as well.

The original patch is from David Woolley but I have modified slightly.

ASTERISK-21094 #close
Reported by: David Woolley
Patches:
      ASTERISK-21094-Patch-1.8-1.txt (license #5737) patch uploaded
      by David Woolley

Change-Id: Ib2b68c6283e227cbeb5fa478b2d0f625dae338ed

The escalator works by creating a set of startup commands in cli.conf
that set up logger channels and issue the debug commands for the
subsystems specified.  If asterisk is running when it is executed,
the same commands will be issued to the running instance.  The original
cli.conf is saved before any changes are made and can be restored by
executing '$prog --reset'.

The log output will be stored in...
$astlogdir/message.$uniqueid
$astlogdir/debug.$uniqueid
$astlogdir/dtmf.$uniqueid
$astlogdir/fax.$uniqueid
$astlogdir/security.$uniqueid
$astlogdir/pjsip_history.$uniqueid
$astlogdir/sip_history.$uniqueid

Some minor tweaks were made to chan_sip, and res_pjsip_history
so their history output could be send to a log channel as packets
are captured.

A minor tweak was also made to manager so events are output to verbose
when "manager set debug on" is issued.

Change-Id: I799f8e5013b86dc5282961b27383d134bf09e543

Issue introduced in b59956a8.  In the non-darwin case libastssl/pj
should be versioned.  This causes the symbol file for this lib
to not be generated.

Change-Id: Ib07ae8c40252813c488e2c1ac6204fd42816dd4c
(cherry picked from commit 54b02791)

* channel.c:ast_sendtext(): Fix T.140 SendText memory leak.

* format_compatibility.c: T.140 RED and T.140 were swapped.

* res_rtp_asterisk.c:rtp_red_init(): Fix ast_format_t140_red ref leak.

* res_rtp_asterisk.c:rtp_red_init(): Fix data race after starting periodic
scheduled red_write().

* res_rtp_asterisk.c: Some other minor misc tweaks.

Change-Id: Ifa27a2e0f8a966b1cf628607c86fc4374b0b88cb

Change-Id: I32e6a589cf9009450e4ff7cb85c07c9d9ef7fe4a

* make_silence() created a malloced silence slin frame without adding a
slin format ref.  When the frame is destroyed it will unref the slin
format that never had a ref added.  Memory corruption is expected to
follow.

* Simplified and fixed counting the number of samples in a frame list for
make_silence().

* Eliminated an unnecessary RAII_VAR associated with the make_silence()
frame.

Change-Id: I47de3f9b92635b7f8b4d72309444d6c0aee6f747

* ast_frisolate() could leak frame format refs on allocation
failures.

* Similified code in ast_frisolate() and code used by
ast_frisolate().

Change-Id: I79566d4d36b3d7801bf0c8294fcd3e9a86a2ed6d

The mechanism used for detecting the maximum log level compiled into the
linked pjproject did not work.  The API call simply stores the requested
level into an integer and does no range checking.  Asterisk was assuming
that there was range checking and limited the new value to the allowable
range.  To get the actual maximum log level compiled into the linked
pjproject we need to get and save off the initial set log level from
pjproject.  This is the maximum log level supported.

* Get and save off the initial log level setting before altering it to the
desired level on startup.  This has to be done by a macro rather than
calling a core function to avoid incorrectly linking pjproject.

* Split the initial log level warning messages to warn if the linked
pjproject cannot support the requested startup level and if it is too low
to get the pjproject buildopts for "pjproject show buildopts".

* Adjust the CLI "pjproject set log level" to check the saved max log
level and to generate normal output messages instead of a warning message.

ASTERISK-26743 #close

Change-Id: I40aa76653e2a1dece66c3f8734594b4f0471cfb4

The 'ari set debug' command has been enhanced to accept 'all' as an
application name.  This allows dumping of all apps even if an app
hasn't registered yet.  To accomplish this, a new global_debug global
variable was added to res/stasis/app.c and new APIs were added to
set and query the value.

'ari set debug' now displays requests and responses as well as events.
This required refactoring the existing debug code.

* The implementation for 'ari set debug' was moved from stasis/cli.{c,h}
  to ari/cli.{c,h}, and stasis/cli.{c,h} were deleted.
* In order to print the body of incoming requests even if a request
  failed, the consumption of the body was moved from the ari stubs
  to ast_ari_callback in res_ari.c and the moustache templates were
  then regenerated.  The body is now passed to ast_ari_invoke and then
  on to the handlers.  This results in code savings since that template
  was inserted multiple times into all the stubs.

An additional change was made to the ao2_str_container implementation
to add partial key searching and a sort function.  The existing cli
code assumed it was already there when it wasn't so the tab completion
was never working.

Change-Id: Ief936f747ce47f1fb14035fbe61152cf766406bf
(cherry picked from commit 1d890874)

This change adds experimental support for providing RTCP
feedback information to codec modules so they can dynamically
change themselves based on conditions.

ASTERISK-26584

Change-Id: Ifd6aa77fb4a7ff546c6025900fc2baf332c31857

It was possible for a frame to be re-inserted into a jitter buffer after it
had been removed from it. A case when this happened was if a frame was read
out of the jitterbuffer, passed to the translation core, and then multiple
frames were returned from said translation core. Upon multiple frames being
returned the first is passed on, but sebsequently "chained" frames are put
back into the read queue. Thus it was possible for a frame to go back into
the jitter buffer where this would cause problems.

This patch adds a flag to frames that are inserted into the channel's read
queue after translation. The abstract jitter buffer code then checks for this
flag and ignores any frames marked as such.

Change-Id: I276c44edc9dcff61e606242f71274265c7779587

The task processor queue reached X scheduled tasks message was originally
intended to get logged only once per task processor to prevent spamming
the log.  This is no longer necessary since high and low water thresholds
can better control when the message is logged.

It is beneficial to generate the warning each time a task processor
reaches the high water level because PJSIP stops processing new requests
while any high water alert is active.  Without this change you would have
to enable at least debug level 3 logging to know about a repeated alert
trigger.

* Made generate the warning message whenever a task is pushed into the
task processor that triggers the high water alert.

* Appended 'again' to the warning for a repeated high water alert trigger.

Change-Id: Iabf75a004f7edaf1e5e8c323099418e667cac999

Adds the ability for extensions to be registered to include filename and
line number so that dialplan show output can show the filename and line
number of a config file responsible for generating a given extension.

This only affects config modules that are written to use the new extension
registering functions. In this patch, that only includes pbx_config, so
extensions registered in extensions.conf and any included extension will
be shown in this manner. Extensions registered in this manner will show
the filename and line number *instead* of the registrar.

ASTERISK-26658 #close
Reported by: Jonathan R. Rose

Change-Id: Ieccc6abccdff34ed5c7da3511fd24972b8f2dd30

* Made not generate strings unless they will actually be used.

ASTERISK-26672

Change-Id: I155fbe7fdff5ce47dfe5326f3baf5446849702c3

ASTERISK-25083

Change-Id: Id54baa57a8dbca84e29f28bcd2ffc0a5ac12d8b2

This is a semi-regression caused by the iostreams change. Prior to
iostreams, HTTP headers were written to a FILE handle using fprintf.
Then the body was written using a call to fwrite(). Because of internal
buffering, the result was that the HTTP headers and body would be sent
out in a single write to the socket.

With the change to iostreams, the HTTP headers are written using
ast_iostream_printf(), which under the hood calls write(). The HTTP body
calls ast_iostream_write(), which also calls write() under the hood.
This results in two separate writes to the socket.

Most HTTP client libraries out there will handle this change just fine.
However, a few of our testsuite tests started failing because of the
change. As a result, in order to reduce frustration for users, this
change alters the HTTP code to write the headers and body in a single
write operation.

ASTERISK-26629 #close
Reported by Joshua Colp

Change-Id: Idc2d2fb3d9b3db14b8631a1e302244fa18b0e518

ast_iostream_printf() attempts first to use a fixed-size buffer to
perform its printf-like operation. If the fixed-size buffer is too
small, then a heap allocation is used instead. The heap allocation in
this case was exactly the length of the string to print. The issue here
is that the ensuing call to vsnprintf() will print a NULL byte in the
final space of the string. This meant that the final character was being
chopped off the string and replaced with a NULL byte. For HTTP in
particular, this caused problems because HTTP publishes the expected
Contact-Length. This meant HTTP was publishing a length one character
larger than what was actually present in the message.

This patch corrects the issue by adding one to the allocation length.

ASTERISK-26629
Reported by Joshua Colp

Change-Id: Ib3c5f41e96833d0415cf000656ac368168add639

Added back in a -g3, and an -O3 when DONT_OPTIMIZE is not set, to
the CFLAGS.  Not sure how they went missing.

Also fixed an uninstall problem where we weren't removing the
symlink from libasteriskpj.so.2 to libasteriskpj.so.  While I was
there, I fixed it for libasteriskssl as well.

Change-Id: I9e00873b1e9082d05b5549d974534b48a2142556

OpenSSL 1.1.0 includes some major changes in the interface. See
https://wiki.openssl.org/index.php/1.1_API_Changes .

Status: Right now there are still a few deprecation notes with OpenSSL
1.1.0. But it's a start.

Changes:
* CRYPTO_LOCK is no longer available. Replace it with its value for now.
  I don't completely understand what it is used for there.
* Remove several functions from libasteriskssl that seem to no longer be
  needed.
* Structures have become opaque and are accesses with accessors.
* ERR_remove_thread_state() no longer needed.
* SSLv2 code now could no longer be used in 1.1.

ASTERISK-26109 #close

Change-Id: I5e29d477d486ca29b6aae0dc2f5dff960c1cb82b

The latest Release candidate fails to create RTP streams when IPv6
is not available. Due to the changes made in September the ast_sockaddr
structure passed around to create these streams is always of AF_INET6
type, causing failure when used for IPv4. This patch adds a utility
function to check for availability of IPv6 and applies such check
at startup to determine how to create the ast_sockaddr structures.

ASTERISK-26617 #close

Change-Id: I627a4e91795e821111e1cda523f083a40d0e0c3e

Use of the new logging is as simple as issuing the new CLI command or
setting the new pjproject.conf option.

Other options that can affect the logging are how you have the pjproject
log levels mapped to Asterisk log types in pjproject.conf and if you have
configured Asterisk to log the DEBUG type messages.  Altering the
pjproject.conf level mapping shouldn't be necessary for most installations
as the default mapping is sensible.  Configuring Asterisk to log the DEBUG
message type is standard practice for collecting debug information.

* Added CLI "pjproject set log level" command to dynamically adjust the
maximum pjproject log message level.

* Added CLI "pjproject show log level" command to see the currently set
maximum pjproject log message level.

* Added pjproject.conf startup section "log_level" option to set the
initial maximum pjproject log message level so all messages could be
captured from initialization.

* Set PJ_LOG_MAX_LEVEL to 6 to compile in all defined logging levels into
bundled pjproject.  Pjproject will use the currently set run time log
level to determine if a log message is generated just like Asterisk
verbose and debug logging levels.

* In log_forwarder(), made always log enabled and mapped pjproject log
messages.  DEBUG mapped log messages are no longer gated by the current
Asterisk debug logging level.

* Removed RAII_VAR() from res_pjproject.c:get_log_level().

ASTERISK-26630 #close

Change-Id: I6dca12979f482ffb0450aaf58db0fe0f6d2e5389

The recent change that made frame deferral into an API had a behavior
change to it. When frame deferral was completed, we would take all of
the deferred frames and queue them all onto the channel in one call to
ast_queue_frame_head(). Before frame deferral was API-ized, places that
performed manual frame deferral would actually take each deferred frame
and queue them onto the channel.

This change in behavior caused the confbridge_recording test to start
failing consistently. Without going too crazily deep into the details,
a channel was getting "stuck" in an ast_safe_sleep(). An AMI redirect
was attempting to break it out of the sleep, but because there were more
frames in the channel read queue than expected, the channel ended up
being unable to break from its sleep loop.

By restoring the behavior of individual frame queuing after deferral,
the test starts passing again.

Note, this points to a potential underlying issue pointing to an
"unbalance" that can occur when queuing multiple frames at once,
and so a follow-up issue is being created to investigate that
possibility.

Change-Id: Ied5dacacda06d343dea751ed5814a03364fe5a7d

The sending codec is switched to the receiving codec and then
is switched back to the best native codec on EVERY receiving RTP packets.
This is because after call of ast_channel_set_rawwriteformat there is call
of ast_set_write_format which calls set_format which sets rawwriteformat
to the best native format.

This patch adds a new function ast_set_write_format_path which set
specific write path on channel and uses this function to switch
the sending codec.

ASTERISK-26603 #close

Change-Id: I5b7d098f8b254ce8f45546e6c36e5d324737f71d

Issue/patch ASTERISK-26587 was inspired by issue ASTERISK-22992
that requested ability to add callerid into app_originate.
Comments in that issue suggested that it was better solved by
adding an option to gosub prior to originating the call.  The
attached patch implements this much like app_dial with two
options one to gosub on the originating channel and one to gosub
on the newly created channel and behaves just like app_dial.
I have tested this patch by adding callerid info to the new
channel and also SIPAddHeader (to e.g. add header to force auto
answer) and confirmed it works.  Have also tested both 'exten'
and 'app' versions of app_originate.

Opened by: dkerr
Patch by: dkerr

Change-Id: I36abc39b58567ffcab4a636ea196ef48be234c57

The asterisk.h header file needs to be included first or else
some things go awry, such as:

implicit declaration of function 'vasprintf'

Change-Id: I981dc2a77a1ba791888e4f1726644d4656c0407c

If a TCP/TLS connection was pending (not accepted and not timed out) during
unload of chan_sip, Asterisk would segfault when trying to send a signal to
a thread whose thread ID hadn't been recorded yet. This commit fixes that by
recording the thread ID before calling the blocking connect() syscall.
This was a regression introduced by 776a1438.

The above wasn't enough to fix the segfault, which was now delayed to the
point where connect() timed out. Therefore, it was necessary to also remove
the SA_RESTART flag from the SIGURG sigaction so that pthread_kill() could be
used to interruput the connect() syscall.
This was a regression introduced by 5d313f51.

ASTERISK-26586 #close

Change-Id: I76fd9d47d56e4264e2629bce8ec15fecba673e7b

When retrieving RTCP stats for PJSIP channels, RTT values are unreliable.
RTT calculation is correct, but the data representation isn't.  RTT is
represented by a 32-bit fixed-point number with the integer part in the
first 16 bits and the fractional part in the last 16 bits.  In order to
get the RTT value, the fractional part is miscalculated, there is an
unnecessary 16 bit shift that causes overflow.  Besides this there is
another mistake, when transforming the integer value to the fixed point
fractional part via bitwise operation, that loses precision.

* RTT fractional part is no longer shifted, avoiding overflow.

* RTT fractional part is transformed to its fixed-point value more
precisely.

* Fixed timeval2ntp() and ntp2timeval() second fraction conversions.

* Fixed NTP timestamp report logging.  The usec was inexplicably
multiplied by 4096.

ASTERISK-26566 #close
Reported by Hector Royo Concepcion

Change-Id: Ie09bdabfee75afb3f1b8ddfd963e5219ada3b96f

Previously, a TLS server socket would only be restarted upon sip reload if the
bind address had changed. This commit adds checking for changes to TLS
parameters like certificate, ciphers, etc. so they get picked up without
requiring a reload of the entire chan_sip module. This does not affect open
connections in any way, but new connections will use the new TLS parameters.
The changes also apply to HTTP and Manager.

ASTERISK-26604 #close

Change-Id: I169e86cefc6dcd627c915134015a6a1ab1aadbe6

libasteriskpj was hard coded to use -lrt but librt is linux specific
so we now use the LIB_RT variable which gets set by configure.

Change-Id: I41148884517e3031f7675a413d524c86e8614694

Fix support of OS's like openBSD that use an older nameser.h,
this change reverts the defines to the older style which on other
systems is found in nameser_compat.h

Tested on openBSD 6.0, Debian 8

ASTERISK-26608 #close

Change-Id: Iffb36caab8c5aa9dece0ce2d009041f7b56cc86a

ASTERISK-26562 #close

Change-Id: Ie6cb0ffc2b8c775639ce7784fe96f4ea00cfa2f8

OpenBSD's 'find' doesn't take the -delete argument so you have to pipe
through 'xargs rm -rf'.

'echo -e' doesn't like \t starting a line. It just prints 't' which
causes the libasteriskpj.exports file to be garbage.  They were just
cosmetic so they were removed.

librt doesn't exist so the link of libasteriskpj.so fails. It's not
actually needed for linux anyway so -lrt was removed from the link.

res_rtp_asterisk was failing to load because of an undefined
DTLS_method. '|| defined(LIBRESSL_VERSION_NUMBER)' was added to the #if
so DTLSv1_method is used instead.

ASTERISK-26608

Change-Id: I926ec95b0b69633231e3ad1d6e803b977272c49c