* acl (named_acl.c)
* cdr
* cel
* ccss
* dnsmgr
* dsp
* enum
* extconfig (config.c)
* features
* http
* indications
* logger
* manager
* plc
* sounds
* udptl

These modules are now loaded at appropriate time by the module loader.
Unlike loadable modules these use AST_MODULE_LOAD_FAILURE on error so
the module loader will abort startup on failure of these modules.

Some of these modules are still initialized or shutdown from outside the
module loader.  logger.c is initialized very early and shutdown very
late, manager.c is initialized by the module loader but is shutdown by
the Asterisk core (too much uses it without holding references).

Change-Id: I371a9a45064f20026c492623ea8062d02a1ab97f

ASTERISK-27722

Change-Id: Ie7b8c30d86cb00a54d6ac4e09e6f28f42d2bd52c

Change-Id: Id52f719078a65c4b2eee7ab99d761eba6b6aed94

This removes references that are no longer needed due to automatic
references created by module dependencies.

In addition this removes most calls to ast_module_check as they were
checking modules which are listed as dependencies.

Change-Id: I332a6e8383d4c72c8e89d988a184ab8320c4872e

* Declare 'requires' and 'enhances' text fields on module info structure.
* Rename 'nonoptreq' to 'optional_modules'.
* Update doxygen comments.

Still need to investigate dependencies among modules I cannot compile.

Change-Id: I3ad9547a0a6442409ff4e352a6d897bef2cc04bf

Change-Id: I828329ecbd252ae8f27a369a046d2b03102b07c6

Add a reference to the calling module when it is active to protect
access to datastore->info.  Remove module references done by
func_periodic_hook as the datastore now handles it.

ASTERISK-25128 #close

Change-Id: I8357a3711e77591d0d1dd8ab4211a7eedd782c89

Be more explicit in what is meant by the master channel to eliminate
misunderstanding.

ASTERISK-23133

Change-Id: I453bcaf4b99404a5a3e345dbf093ac6c1afcfc72

Change-Id: I873c1c6d00f447269bd841494459efccdd2c19c0

This module uses AST_DEFINE_APP_ARGS_TYPE to define struct's instead of
directly using AST_DECLARE_APP_ARGS.  Initialize the variables declared
in this way.

Change-Id: If97fbdd8d63a204e2efd498a192effc14e90fb31

Setting channel variables with the AMI Originate action caused a deadlock
when you set CDR(amaflags) or CDR(accountcode).  This path has the channel
locked when the CDR function is called.  The CDR function then
synchronously passes the job to a stasis thread.  The stasis handling
function then attempts to lock the channel.  Deadlock results.

* Avoid deadlock by making the CDR function handle setting amaflags and
accountcode directly on the channel rather than passing it off to the CDR
processing code under a stasis thread to do it.

* Made the CHANNEL function and the CDR function process amaflags the same
way.

* Fixed referencing the wrong message type in cdr_prop_write().

ASTERISK-27460

Change-Id: I5eacb47586bc0b8f8ff76a19bd92d1dc38b75e8f

This adds menuselect dependencies for modules that use symbols of other
modules.

ASTERISK-27390

Change-Id: Ia2d2849f5b87a72af7324a82edc3f283eafb5385

Once an Optional API module is loaded it should stay loaded.  Unloading
an optional API module runs the risk of a crash if something else is
using it.  This patch causes all optional API providers to tell the
module loader not to unload except at shutdown.

ASTERISK-27389

Change-Id: Ia07786fe655681aec49cc8d3d96e06483b11f5e6

Some endpoints do not like a stream being reused for a new
media stream. The frame/jitterbuffer can rely on underlying
attributes of the media stream in order to order the packets.
When a new stream takes its place without any notice the
buffer can get confused and the media ends up getting dropped.

This change uses the SSRC change to determine that a new source
is reusing an existing stream and then bridge_softmix renegotiates
each participant such that they see a new media stream. This
causes the frame/jitterbuffer to start fresh and work as expected.

ASTERISK-27277

Change-Id: I30ccbdba16ca073d7f31e0e59ab778c153afae07

Fixes ${CDR(...,u)} when used in cdr_custom.conf

ASTERISK-27165 #close

Change-Id: Ia4e0b6ba93e03d27886354c279737790e2cd6a83

An admin can configure app_minivm with an externnotify program to be run
when a voicemail is received.  The app_minivm application MinivmNotify
uses ast_safe_system() for this purpose which is vulnerable to command
injection since the Caller-ID name and number values given to externnotify
can come from an external untrusted source.

* Add ast_safe_execvp() function.  This gives modules the ability to run
external commands with greater safety compared to ast_safe_system().
Specifically when some parameters are filled by untrusted sources the new
function does not allow malicious input to break argument encoding.  This
may be of particular concern where CALLERID(name) or CALLERID(num) may be
used as a parameter to a script run by ast_safe_system() which could
potentially allow arbitrary command execution.

* Changed app_minivm.c:run_externnotify() to use the new ast_safe_execvp()
instead of ast_safe_system() to avoid command injection.

* Document code injection potential from untrusted data sources for other
shell commands that are under user control.

ASTERISK-27103

Change-Id: I7552472247a84cde24e1358aaf64af160107aef1

A regression was introduced in 12 where passing an empty value
to the CDR dialplan function was not longer allowed. This
change returns to the behavior of 11 where it is permitted.

ASTERISK-26173

Change-Id: I3f148203b54ec088007e29e30005a5de122e51c5

In all non-pbx modules, AST_MODULE_LOAD_FAILURE has been changed
to AST_MODULE_LOAD_DECLINE.  This prevents asterisk from exiting
if a module can't be loaded.  If the user wishes to retain the
FAILURE behavior for a specific module, they can use the "require"
or "preload-require" keyword in modules.conf.

A new API was added to logger: ast_is_logger_initialized().  This
allows asterisk.c/check_init() to print to the error log once the
logger subsystem is ready instead of just to stdout.  If something
does fail before the logger is initialized, we now print to stderr
instead of stdout.

Change-Id: I5f4b50623d9b5a6cb7c5624a8c5c1274c13b2b25

* Added CHANNEL(callid) to retrieve the call identifier log tag associated
with the channel.  Dialplan now has access to the call log search key
associated with the channel so it can be saved in case there is a problem
with the call.

ASTERISK-26878

Change-Id: I2c97ebd928b6f3c5bc80c5729e4d3c07f453049f

During module loading of func_devstate, Asterisk emits the current
device state of all Custom device states currently stored in the AstDB.
This was erroneously including a new line character ('\n') to the end of
the device state, causing two new lines to be emitted in
DeviceStateChange AMI events.

Note that this only happened for those device state changes that
occurred during startup. Regular device state changes for Custom device
states are handled elsewhere, and did not have the newline.

ASTERISK-26643 #close
Reported by: Roman Bedros
Tested by: Matt Jordan
patches:
  ami_devstate.diff uploaded by Roman Bedros (License 6842)

Change-Id: I1f4c02fc79c448d43bf725f5039c83d9611d7d93

This change adds a few things to facilitate stream topology changing:

1. Control frame types have been added for use by the channel driver
to notify the application that the channel wants to change the stream
topology or that a stream topology change has been accepted. They are
also used by the indicate interface to the channel that the application
uses to indicate it wants to do the same.

2. Legacy behavior has been adopted in ast_read() such that if a
channel requests a stream topology change it is denied automatically
and the current stream topology is preserved if the application is
not capable of handling streams.

Tests have also been written which confirm the multistream and
non-multistream behavior.

ASTERISK-26839

Change-Id: Ia68ef22bca8e8457265ca4f0f9de600cbcc10bc9

* app_minivm: Use built-in completion facilities to complete optional
arguments.

* app_voicemail: Use built-in completion facilities to complete
optional arguments.

* app_confbridge: Add missing colons after 'Usage' text.

* chan_alsa: Use built-in completion facilities to complete optional
arguments.

* chan_sip: Use built-in completion facilities to complete optional
arguments. Add completions for 'load' for 'sip show user', 'sip show
peer', and 'sip qualify peer.'

* chan_skinny: Correct and extend completions for 'skinny reset' and
'skinny show line.'

* func_odbc: Correct completions for 'odbc read' and 'odbc write'

* main/astmm: Use built-in completion facilities to complete arguments
for 'memory' commands.

* main/bridge: Correct completions for 'bridge kick.'

* main/ccss: Use built-in completion facilities to complete arguments
for 'cc cancel' command.

* main/cli: Add 'all' completion for 'channel request hangup.' Correct
completions for 'core set debug channel.' Correct completions for 'core
show calls.'

* main/pbx_app: Remove redundant completions for 'core show
applications.'

* main/pbx_hangup_handler: Remove unused completions for 'core show
hanguphandlers all.'

* res_sorcery_memory_cache: Add completion for 'reload' argument of
'sorcery memory cache stale' and properly implement.

Change-Id: Iee58c7392f6fec34ad9d596109117af87697bbca

In Asterisk 11, if the 'Originate' AMI command failed to connect the provided
Channel while in extension mode, a 'failed' extension would be looked up and
run. This was, I believe, unintentionally removed in 51b6c496. This patch
restores that behavior.

This also adds an enum for the various 'synchronous' modes in an attempt to
make them meaningful.

ASTERISK-26115 #close
Reported by: Nasir Iqbal

Change-Id: I8afbd06725e99610e02adb529137d4800c05345d

This change adds experimental support for providing RTCP
feedback information to codec modules so they can dynamically
change themselves based on conditions.

ASTERISK-26584

Change-Id: Ifd6aa77fb4a7ff546c6025900fc2baf332c31857

Feeding LISTFILTER an empty variable results in an invalid ERROR message.
Earlier changes made the message useless because we can no longer tell if
the variable is empty or does not exist.  It is valid to try to remove a
value from an empty list just as it is valid to try to remove a value that
is not in a non-empty list.

* Removed the outdated ERROR message.

* Added more test cases to the LISTFILTER unit test.

Change-Id: Ided9040e6359c44a335ef54e02ef5950a1863134

ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.

Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename

This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled.  This variable was only used in lock.c so it
is now initialized in that file only.

ASTERISK-26480 #close

Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966

This updates func_channel.c and main/message.c to use a generic xpointer
include instead of including info from each channel driver.  Now the
name attribute of info is CHANNEL or CHANNEL_EXAMPLES to be included in
documentation for func_channel.  Setting the name attribute of info to
MessageToInfo or MessageFromInfo causes it to be included in the
MessageSend application and AMI action.

Change-Id: I89fd8276a3250824241a618009714267d3a8d1ea

* Following the example of the PJSIP channel driver, the channel
  technology specific documentation has been moved to the respective
  channel drivers that provide that functionality. This has the benefit
  of locating the documentation of items with those modules that provide
  it.

* Examples of using the CHANNEL function for both standard items as well
  as for PJSIP have been added.

* The 'max_forwards' standard item has been documented.

Change-Id: Ifaa79a232c8ac99cf8da6ef6cc7815d398b1b79b

In a timeval, tv_usec is defined as a suseconds_t, which could be
different underlying types on different platforms. Instead of trying to
scanf directly into the timeval, scanf into a long int, then copy that
into the timeval.

Change-Id: I29f22d049d3f7746b6c0cc23fbf4293bdaa5eb95

If AST_TEST_DEFINE is not conditional to TEST_FRAMEWORK it produces dead
code.  This places all existing unit tests into a conditional block if
they weren't already.

ASTERISK-26211 #close

Change-Id: I8ef83ee11cbc991b07b7a37ecb41433e8c734686

The func_odbc module was modified to ensure that the
previous behavior of using a single database connection
was maintained. This was done by getting a single database
connection and holding on to it. With the new multiple
connection support in res_odbc this will actually starve
every other thread from getting access to the database as
it also maintains the previous behavior of having only
a single database connection.

This change disables the func_odbc specific behavior if
the res_odbc module is running with only a single database
connection active. The connection is only kept for the
duration of the request.

ASTERISK-26177 #close

Change-Id: I9bdbd8a300fb3233877735ad3fd07bce38115b7f

gcc 6.1.1 caught a few more issues.
Made sure the unit tests still pass for the func_env and stdtime
issues.

ASTERISK-26157 #close

Change-Id: I6664d8f34a45bc1481d2a854481c7878b0c1cf8e

func_odbc was changed in Asterisk 13.9.0
to make func_odbc use a single database connection per DSN
because of reported bug ASTERISK-25938
with MySQL/MariaDB LAST_INSERT_ID().

This is drawback in performance when func_odbc is used
very often in dialplan.

Single database connection should be optional.

ASTERISK-26010

Change-Id: I7091783a7150252de8eeb455115bd00514dfe843

A recent change to func_odbc made it so that a single connection was
maintained per DSN. The problem was that the code was optimistic about
the health of the connection after initially opening it and did nothing
to re-connect in case the connection had died.

This change adds a check before executing a query to ensure that the
connection to the database is still up and running.

ASTERISK-25963 #close
Reported by Ross Beer

Change-Id: Id33c86eb04ff48ca088bb2e3086c27b3b683491d

res_odbc was changed in Asterisk 13.8.0 to remove connection management,
opting instead to let unixodbc maintain open connections and return
those to Asterisk as requested.

This was a boon for realtime, since it meant that multiple threads could
potentially run parallel queries since they could each be using their
own database connections.

However, on the user-facing side, func_odbc, there were some inherent
behaviors being relied on that no longer hold true after the change.
One such reported behavior was that MySQL's LAST_INSERTED_ID() works
per-connection. This means that if Asterisk uses separate connections
for every database operation, whereas before it used one connection for
everything, we have broken expectations and functionality.

The fix provided in this patch is to make func_odbc use a single
database connection per DSN. This way, user-facing database usage will
have the same behavior as it did pre-13.8.0. However, realtime, which is
the real workhorse of database interaction, will continue to let
unixodbc manage connections.

ASTERISK-25938 #close
Reported by Edwin Vandamme

Change-Id: Iac961fe79154c6211569afcdfec843c0c24c46dc

This patch adds a write option to the CURL dialplan function, allowing it to
CURL files and store them locally. The value 'written' to the CURL URL
specifies the location on disk to store the file. As an example:

same => n,Set(CURL(http://1.1.1.1/foo.wav)=/tmp/foo.wav)

Would retrieve the file foo.wav from the remote server and store it in the
/tmp directory.

Due to the potentially dangerous nature of this function call, APIs are
forbidden from using the write functionality unless live_dangerously is set
to True in asterisk.conf.

ASTERISK-25652 #close

Change-Id: I44f4ad823d7d20f04ceaad3698c5c7f653c41b0d

The encryption code for AES_ENCRYPT evaluates the length of the data to
be encoded in base64 using strlen. The data is binary, thus the length
of it can be underestimated at the first NULL character.
Reuse the write pointer offset to evaluate it, instead.

ASTERISK-25857 #close

Change-Id: If686b5d570473eb926693c73461177b35b13b186

Change-Id: I6e8d39b0711110a4bceafa652e58b30465e28386

ASTERISK-25272 #close
Reported by: Etienne Lessard
patches:
 AST-25272.patch submitted by Etienne Lessard (license #6394)

Change-Id: Id75ad202300960a1e91afe15e319d992936ecc17

Asterisk by default will create a single database connection and share
it among all threads that attempt to access the database. In previous
versions of Asterisk, this was tolerable, because the most used channel
driver, chan_sip, mostly accessed the database from a single thread.
With PJSIP, however, many threads may be attempting to perform database
operations, and there is the potential for many more database accesses,
meaning the concurrency is a horrible bottleneck if only one connection
is shared.

Asterisk has a connection pooling facility built into it, but the
implementation has flaws. For one, there is a strict limit on the number
of simultaneous connections that could be made to the database. Anything
beyond the maximum would result in a failed operation. Attempting to
predict what the maximum should be is nearly impossible even for someone
intimately familiar with Asterisk's threading model. In addition, use of
transactions in the dialplan can cause some severe bugs if connection
pooling is enabled.

This commit seeks to fix the concurrency problem by removing all
connection management code from Asterisk and leaving that to the
underlying unixODBC code instead. Now, Asterisk does not share a single
connection, nor does it try to maintain a connection pool. Instead, all
Asterisk ever does is request a connection from unixODBC and allow
unixODBC to either allocate those connections or retrieve them from a
pool.

Doing this has a bit of a ripple effect. For one, since connections are
not long-lived objects, several of the safeguards that previously
existed have been removed. We don't have to worry about trying to use a
connection that has gone stale. In every case, when we request a
connection, it has just been made and we don't need to perform any
sanity checks to be sure it's still active.

Another major player affected by this change is transactions.
Transactions and their respective connections were so tightly coupled
that it was almost pornographic. This code change moves
transaction-related code to its own file separate from the core ODBC
functionality. This way, the core of ODBC does not even have to know
that transactions exist.

In making this large change, I had to look at a lot of code and
understand it. When making this change, I discovered several places
where the behavior is definitely not ideal, but it seemed outside the
scope of this change to be fixing it. Instead, any place where I saw
some sort of room for improvement has had a XXX comment added explaining
what could be altered to improve it.

Change-Id: I37a84def5ea4ddf93868ce8105f39de078297fbf