This adds menuselect dependencies for modules that use symbols of other
modules.

ASTERISK-27390

Change-Id: Ia2d2849f5b87a72af7324a82edc3f283eafb5385

Once an Optional API module is loaded it should stay loaded.  Unloading
an optional API module runs the risk of a crash if something else is
using it.  This patch causes all optional API providers to tell the
module loader not to unload except at shutdown.

ASTERISK-27389

Change-Id: Ia07786fe655681aec49cc8d3d96e06483b11f5e6

Some endpoints do not like a stream being reused for a new
media stream. The frame/jitterbuffer can rely on underlying
attributes of the media stream in order to order the packets.
When a new stream takes its place without any notice the
buffer can get confused and the media ends up getting dropped.

This change uses the SSRC change to determine that a new source
is reusing an existing stream and then bridge_softmix renegotiates
each participant such that they see a new media stream. This
causes the frame/jitterbuffer to start fresh and work as expected.

ASTERISK-27277

Change-Id: I30ccbdba16ca073d7f31e0e59ab778c153afae07

Fixes ${CDR(...,u)} when used in cdr_custom.conf

ASTERISK-27165 #close

Change-Id: Ia4e0b6ba93e03d27886354c279737790e2cd6a83

An admin can configure app_minivm with an externnotify program to be run
when a voicemail is received.  The app_minivm application MinivmNotify
uses ast_safe_system() for this purpose which is vulnerable to command
injection since the Caller-ID name and number values given to externnotify
can come from an external untrusted source.

* Add ast_safe_execvp() function.  This gives modules the ability to run
external commands with greater safety compared to ast_safe_system().
Specifically when some parameters are filled by untrusted sources the new
function does not allow malicious input to break argument encoding.  This
may be of particular concern where CALLERID(name) or CALLERID(num) may be
used as a parameter to a script run by ast_safe_system() which could
potentially allow arbitrary command execution.

* Changed app_minivm.c:run_externnotify() to use the new ast_safe_execvp()
instead of ast_safe_system() to avoid command injection.

* Document code injection potential from untrusted data sources for other
shell commands that are under user control.

ASTERISK-27103

Change-Id: I7552472247a84cde24e1358aaf64af160107aef1

A regression was introduced in 12 where passing an empty value
to the CDR dialplan function was not longer allowed. This
change returns to the behavior of 11 where it is permitted.

ASTERISK-26173

Change-Id: I3f148203b54ec088007e29e30005a5de122e51c5

In all non-pbx modules, AST_MODULE_LOAD_FAILURE has been changed
to AST_MODULE_LOAD_DECLINE.  This prevents asterisk from exiting
if a module can't be loaded.  If the user wishes to retain the
FAILURE behavior for a specific module, they can use the "require"
or "preload-require" keyword in modules.conf.

A new API was added to logger: ast_is_logger_initialized().  This
allows asterisk.c/check_init() to print to the error log once the
logger subsystem is ready instead of just to stdout.  If something
does fail before the logger is initialized, we now print to stderr
instead of stdout.

Change-Id: I5f4b50623d9b5a6cb7c5624a8c5c1274c13b2b25

* Added CHANNEL(callid) to retrieve the call identifier log tag associated
with the channel.  Dialplan now has access to the call log search key
associated with the channel so it can be saved in case there is a problem
with the call.

ASTERISK-26878

Change-Id: I2c97ebd928b6f3c5bc80c5729e4d3c07f453049f

During module loading of func_devstate, Asterisk emits the current
device state of all Custom device states currently stored in the AstDB.
This was erroneously including a new line character ('\n') to the end of
the device state, causing two new lines to be emitted in
DeviceStateChange AMI events.

Note that this only happened for those device state changes that
occurred during startup. Regular device state changes for Custom device
states are handled elsewhere, and did not have the newline.

ASTERISK-26643 #close
Reported by: Roman Bedros
Tested by: Matt Jordan
patches:
  ami_devstate.diff uploaded by Roman Bedros (License 6842)

Change-Id: I1f4c02fc79c448d43bf725f5039c83d9611d7d93

This change adds a few things to facilitate stream topology changing:

1. Control frame types have been added for use by the channel driver
to notify the application that the channel wants to change the stream
topology or that a stream topology change has been accepted. They are
also used by the indicate interface to the channel that the application
uses to indicate it wants to do the same.

2. Legacy behavior has been adopted in ast_read() such that if a
channel requests a stream topology change it is denied automatically
and the current stream topology is preserved if the application is
not capable of handling streams.

Tests have also been written which confirm the multistream and
non-multistream behavior.

ASTERISK-26839

Change-Id: Ia68ef22bca8e8457265ca4f0f9de600cbcc10bc9

* app_minivm: Use built-in completion facilities to complete optional
arguments.

* app_voicemail: Use built-in completion facilities to complete
optional arguments.

* app_confbridge: Add missing colons after 'Usage' text.

* chan_alsa: Use built-in completion facilities to complete optional
arguments.

* chan_sip: Use built-in completion facilities to complete optional
arguments. Add completions for 'load' for 'sip show user', 'sip show
peer', and 'sip qualify peer.'

* chan_skinny: Correct and extend completions for 'skinny reset' and
'skinny show line.'

* func_odbc: Correct completions for 'odbc read' and 'odbc write'

* main/astmm: Use built-in completion facilities to complete arguments
for 'memory' commands.

* main/bridge: Correct completions for 'bridge kick.'

* main/ccss: Use built-in completion facilities to complete arguments
for 'cc cancel' command.

* main/cli: Add 'all' completion for 'channel request hangup.' Correct
completions for 'core set debug channel.' Correct completions for 'core
show calls.'

* main/pbx_app: Remove redundant completions for 'core show
applications.'

* main/pbx_hangup_handler: Remove unused completions for 'core show
hanguphandlers all.'

* res_sorcery_memory_cache: Add completion for 'reload' argument of
'sorcery memory cache stale' and properly implement.

Change-Id: Iee58c7392f6fec34ad9d596109117af87697bbca

In Asterisk 11, if the 'Originate' AMI command failed to connect the provided
Channel while in extension mode, a 'failed' extension would be looked up and
run. This was, I believe, unintentionally removed in 51b6c496. This patch
restores that behavior.

This also adds an enum for the various 'synchronous' modes in an attempt to
make them meaningful.

ASTERISK-26115 #close
Reported by: Nasir Iqbal

Change-Id: I8afbd06725e99610e02adb529137d4800c05345d

This change adds experimental support for providing RTCP
feedback information to codec modules so they can dynamically
change themselves based on conditions.

ASTERISK-26584

Change-Id: Ifd6aa77fb4a7ff546c6025900fc2baf332c31857

Feeding LISTFILTER an empty variable results in an invalid ERROR message.
Earlier changes made the message useless because we can no longer tell if
the variable is empty or does not exist.  It is valid to try to remove a
value from an empty list just as it is valid to try to remove a value that
is not in a non-empty list.

* Removed the outdated ERROR message.

* Added more test cases to the LISTFILTER unit test.

Change-Id: Ided9040e6359c44a335ef54e02ef5950a1863134

ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.

Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename

This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled.  This variable was only used in lock.c so it
is now initialized in that file only.

ASTERISK-26480 #close

Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966

This updates func_channel.c and main/message.c to use a generic xpointer
include instead of including info from each channel driver.  Now the
name attribute of info is CHANNEL or CHANNEL_EXAMPLES to be included in
documentation for func_channel.  Setting the name attribute of info to
MessageToInfo or MessageFromInfo causes it to be included in the
MessageSend application and AMI action.

Change-Id: I89fd8276a3250824241a618009714267d3a8d1ea

* Following the example of the PJSIP channel driver, the channel
  technology specific documentation has been moved to the respective
  channel drivers that provide that functionality. This has the benefit
  of locating the documentation of items with those modules that provide
  it.

* Examples of using the CHANNEL function for both standard items as well
  as for PJSIP have been added.

* The 'max_forwards' standard item has been documented.

Change-Id: Ifaa79a232c8ac99cf8da6ef6cc7815d398b1b79b

In a timeval, tv_usec is defined as a suseconds_t, which could be
different underlying types on different platforms. Instead of trying to
scanf directly into the timeval, scanf into a long int, then copy that
into the timeval.

Change-Id: I29f22d049d3f7746b6c0cc23fbf4293bdaa5eb95

If AST_TEST_DEFINE is not conditional to TEST_FRAMEWORK it produces dead
code.  This places all existing unit tests into a conditional block if
they weren't already.

ASTERISK-26211 #close

Change-Id: I8ef83ee11cbc991b07b7a37ecb41433e8c734686

The func_odbc module was modified to ensure that the
previous behavior of using a single database connection
was maintained. This was done by getting a single database
connection and holding on to it. With the new multiple
connection support in res_odbc this will actually starve
every other thread from getting access to the database as
it also maintains the previous behavior of having only
a single database connection.

This change disables the func_odbc specific behavior if
the res_odbc module is running with only a single database
connection active. The connection is only kept for the
duration of the request.

ASTERISK-26177 #close

Change-Id: I9bdbd8a300fb3233877735ad3fd07bce38115b7f

gcc 6.1.1 caught a few more issues.
Made sure the unit tests still pass for the func_env and stdtime
issues.

ASTERISK-26157 #close

Change-Id: I6664d8f34a45bc1481d2a854481c7878b0c1cf8e

func_odbc was changed in Asterisk 13.9.0
to make func_odbc use a single database connection per DSN
because of reported bug ASTERISK-25938
with MySQL/MariaDB LAST_INSERT_ID().

This is drawback in performance when func_odbc is used
very often in dialplan.

Single database connection should be optional.

ASTERISK-26010

Change-Id: I7091783a7150252de8eeb455115bd00514dfe843

A recent change to func_odbc made it so that a single connection was
maintained per DSN. The problem was that the code was optimistic about
the health of the connection after initially opening it and did nothing
to re-connect in case the connection had died.

This change adds a check before executing a query to ensure that the
connection to the database is still up and running.

ASTERISK-25963 #close
Reported by Ross Beer

Change-Id: Id33c86eb04ff48ca088bb2e3086c27b3b683491d

res_odbc was changed in Asterisk 13.8.0 to remove connection management,
opting instead to let unixodbc maintain open connections and return
those to Asterisk as requested.

This was a boon for realtime, since it meant that multiple threads could
potentially run parallel queries since they could each be using their
own database connections.

However, on the user-facing side, func_odbc, there were some inherent
behaviors being relied on that no longer hold true after the change.
One such reported behavior was that MySQL's LAST_INSERTED_ID() works
per-connection. This means that if Asterisk uses separate connections
for every database operation, whereas before it used one connection for
everything, we have broken expectations and functionality.

The fix provided in this patch is to make func_odbc use a single
database connection per DSN. This way, user-facing database usage will
have the same behavior as it did pre-13.8.0. However, realtime, which is
the real workhorse of database interaction, will continue to let
unixodbc manage connections.

ASTERISK-25938 #close
Reported by Edwin Vandamme

Change-Id: Iac961fe79154c6211569afcdfec843c0c24c46dc

This patch adds a write option to the CURL dialplan function, allowing it to
CURL files and store them locally. The value 'written' to the CURL URL
specifies the location on disk to store the file. As an example:

same => n,Set(CURL(http://1.1.1.1/foo.wav)=/tmp/foo.wav)

Would retrieve the file foo.wav from the remote server and store it in the
/tmp directory.

Due to the potentially dangerous nature of this function call, APIs are
forbidden from using the write functionality unless live_dangerously is set
to True in asterisk.conf.

ASTERISK-25652 #close

Change-Id: I44f4ad823d7d20f04ceaad3698c5c7f653c41b0d

The encryption code for AES_ENCRYPT evaluates the length of the data to
be encoded in base64 using strlen. The data is binary, thus the length
of it can be underestimated at the first NULL character.
Reuse the write pointer offset to evaluate it, instead.

ASTERISK-25857 #close

Change-Id: If686b5d570473eb926693c73461177b35b13b186

Change-Id: I6e8d39b0711110a4bceafa652e58b30465e28386

ASTERISK-25272 #close
Reported by: Etienne Lessard
patches:
 AST-25272.patch submitted by Etienne Lessard (license #6394)

Change-Id: Id75ad202300960a1e91afe15e319d992936ecc17

Asterisk by default will create a single database connection and share
it among all threads that attempt to access the database. In previous
versions of Asterisk, this was tolerable, because the most used channel
driver, chan_sip, mostly accessed the database from a single thread.
With PJSIP, however, many threads may be attempting to perform database
operations, and there is the potential for many more database accesses,
meaning the concurrency is a horrible bottleneck if only one connection
is shared.

Asterisk has a connection pooling facility built into it, but the
implementation has flaws. For one, there is a strict limit on the number
of simultaneous connections that could be made to the database. Anything
beyond the maximum would result in a failed operation. Attempting to
predict what the maximum should be is nearly impossible even for someone
intimately familiar with Asterisk's threading model. In addition, use of
transactions in the dialplan can cause some severe bugs if connection
pooling is enabled.

This commit seeks to fix the concurrency problem by removing all
connection management code from Asterisk and leaving that to the
underlying unixODBC code instead. Now, Asterisk does not share a single
connection, nor does it try to maintain a connection pool. Instead, all
Asterisk ever does is request a connection from unixODBC and allow
unixODBC to either allocate those connections or retrieve them from a
pool.

Doing this has a bit of a ripple effect. For one, since connections are
not long-lived objects, several of the safeguards that previously
existed have been removed. We don't have to worry about trying to use a
connection that has gone stale. In every case, when we request a
connection, it has just been made and we don't need to perform any
sanity checks to be sure it's still active.

Another major player affected by this change is transactions.
Transactions and their respective connections were so tightly coupled
that it was almost pornographic. This code change moves
transaction-related code to its own file separate from the core ODBC
functionality. This way, the core of ODBC does not even have to know
that transactions exist.

In making this large change, I had to look at a lot of code and
understand it. When making this change, I discovered several places
where the behavior is definitely not ideal, but it seemed outside the
scope of this change to be fixing it. Instead, any place where I saw
some sort of room for improvement has had a XXX comment added explaining
what could be altered to improve it.

Change-Id: I37a84def5ea4ddf93868ce8105f39de078297fbf

When CDRs were refactored, func_cdr's ability to report high precision values
for duration and billsec (the 'f' option) was broken. This was due to func_cdr
incorrectly interpreting the duration/billsec values provided by the CDR engine
in milliseconds, as opposed to seconds. Since the CDR engine only provides
duration and billsec in seconds, and does not expose either attribute with
sufficient precision to merely pass back the underlying value, this patch fixes
the bug by re-calculating duration and billsec with microsecond precision based
on the start/answer/end times on the CDR.

ASTERISK-25179 #close

Change-Id: I8bc63822b496537a5bf80baf6102c06206bee841

Adding help text documentation for:
* hangupsource
* appname
* appdata
* exten
* context
* channame
* uniqueid
* linkedid

ASTERISK-24097 #close
Reported by: Steven T. Wheeler
Tested by: Rusty Newton

Change-Id: Ib94b00568b0433987df87d5b67ea529b5905754d

ASTERISK-25533 #close

Change-Id: Ie1a9d1a6511b3f1a56b93d04475fbf8a4e40010a

CALLERPRES() says that it's deprecated in favor of CALLERID(num-pres)
and CALLERID(name-pres).  But for channel driver that don't make a
distinction between the two (e.g. SIP), it makes more sense to get/set
both at once.  This change reveals the availability of CALLERID(pres),
CONNECTEDLINE(pres), REDIRECTING(orig-pres), REDIRECTING(to-pres) and
REDIRECTING(from-pres).

ASTERISK-25373 #close

Change-Id: I5614ae4ab7d3bbe9c791c1adf147e10de8698d7a

When ab803ec3 was committed, it accidentally forgot to actually *add* the
HOLD_INTERCEPT function. This highlights two interesting points:
* Gerrit forces you to put the patch as it is going to into the repo up for
  review, which Review Board did not. Yay Gerrit.
* No one apparently bothered to use this feature, or else they don't know about
  it. I'm going to go with the latter explanation.

ASTERISK-24922

Change-Id: Ida38278f259dd07c334a36f9b7d5475b5db72396

Return AST_PRESENCE_NOT_SET when CustomPresence AstDB key does not
exist, i.e. when a new CustomPresence is added in the dialplan.

ASTERISK-25400 #close
Reported by: Andrew Nagy

Change-Id: I6fb17b16591b5a55fbffe96f3994ec26b1b1723a

 * In sip.conf.sample fix sentence where we said that WS or WSS are supported
   transports for use in an outbound register definition. They are not
   supported in that case.
 * In func_cdr.c made it clear that the Disable option for CDR_PROP can be used
   to enable CDR on a channel.

ASTERISK-24867 #close
Reported by: Rusty Newton

ASTERISK-24853 #close
Reported by: PSDK

Change-Id: I3d698bc6302b9d00a0a995b5c4ad9a42d69b48ca

ASTERISK-25162 #close

Change-Id: Id79aa3c6fe490016ee98efc97ac4c1d3f461f97e

These modules save a pointer to the context they create on load, and
use that pointer to destroy the context at unload.  It is not safe
to save this pointer, it is replaced during load of pbx_config,
pbx_lua or pbx_ael.

This change causes the modules to pass NULL to ast_context_destroy,
a safer way to perform the unregistration since it does not use
a pointer that could become invalid.

ASTERISK-25085 #close
Reported by: Corey Farrell

Change-Id: I6a00ec8e38046058f97dc703e1adcde9bf517835

Change-Id: Icf88f9f861c6b2a16e5f626ff25795218a6f2723

The "Add qualify_timeout processing and eventing" patch introduced
an issue where contacts that had qualify_frequency set to 0 were
showing Unavailable instead Unknown.  This patch checks for
qualify_frequency=0 and create an "Unknown"  contact_status
with an RTT = 0.

Previously, the lack of contact_status implied Unknown but since
we're now changing endpoint state based on contact_status, I've
had to add new UNKNOWN status so that changes could trigger the
appropriate contact_status observers.

ASTERISK-24977: #close

Change-Id: Ifcbc01533ce57f0e4e584b89a395326e098b8fe7