When Asterisk is connected and used with a database the response
time of the database can cause problems in Asterisk if it is long.
Normally the only way to see this problem would be to retrieve a
backtrace from Asterisk and examine where things are blocked, or
examine the database to see if there is any indication of a
problem.

This change adds some basic query logging to make it easier to
investigate such a problem. When logging is enabled res_odbc will
now keep track of the number of queries executed, as well as the
query that has taken the longest time to execute. There is also
an option which will cause a WARNING message to be output if a
query takes longer than a configurable amount of time to execute.

This makes it easier and clearer for users that their database may
be experiencing a problem that could impact Asterisk.

ASTERISK-28277

Change-Id: I173cf4928b10754478a6a8c27dfa96ede0f058a6

* acl (named_acl.c)
* cdr
* cel
* ccss
* dnsmgr
* dsp
* enum
* extconfig (config.c)
* features
* http
* indications
* logger
* manager
* plc
* sounds
* udptl

These modules are now loaded at appropriate time by the module loader.
Unlike loadable modules these use AST_MODULE_LOAD_FAILURE on error so
the module loader will abort startup on failure of these modules.

Some of these modules are still initialized or shutdown from outside the
module loader.  logger.c is initialized very early and shutdown very
late, manager.c is initialized by the module loader but is shutdown by
the Asterisk core (too much uses it without holding references).

Change-Id: I371a9a45064f20026c492623ea8062d02a1ab97f

Checking option_debug directly is incorrect as it ignores file/module
specific debug settings.  This system-wide change replaces nearly all
direct checks for option_debug with the DEBUG_ATLEAST macro.

Change-Id: Ic342d4799a945dbc40ac085ac142681094a4ebf0

ASTERISK-27722

Change-Id: Ie7b8c30d86cb00a54d6ac4e09e6f28f42d2bd52c

Asterisk's makefile for menuselect has a very simple source file parsing
script that looks for AST_MODULE_INFO lines to extract the quoted string
as a module description.  If it does not find a quoted string it uses the
whole line as the description.

Change-Id: I80f13a63818e4e28d683639a94a4dfaea405c1d5

Change-Id: I873c1c6d00f447269bd841494459efccdd2c19c0

Beanstalkd is a simple to use job queue. It provides a means to
create multiple job queues called "tubes". Each tube can store
multiple jobs, with varying priorities with the queue. Queue
processing is available via a simple TCP socket or via well defined
libraries, avaialble at
https://github.com/kr/beanstalkd/wiki/client-libraries

This module is based upon the beanstalk-client library, available
for download at: https://github.com/deepfryed/beanstalk-client

This module currently doesn't support user defined events.

Change-Id: Ic3a087faeeac045d69a2a018e60e29831ddb95ab

Enables runtime configuration of busy_timeout for sqlite databases.
Default timeout remains 1000ms.

ASTERISK-27014 #close

Change-Id: I8921a3aac3c335843be4cb17d2dd0a5c157a36da

When a column is of type timestamp, the fraction part of the event
field's seconds was frequently parsed incorrectly especially if
there were leading zeros.  For instance "2017-05-23 23:55:03.023"
would be parsed into an int as "23" then when the timestamp was
formatted again to be inserted into the database column it'd be
"2017-05-23 23:55:03.23" which is now 230 milliseconds instead of
23 milliseconds.  "03.000001" would be transformed to "03.1", etc.

* If the event field is 'eventtime' and the db column is timestamp,
  then existing processing has already correctly formatted the
  timestamp so now we simply use it rather than parsing it and
  re-printing it. This is the most common use case anyway.

* If the event field is other than 'eventtime' and the db column
  is timestamp, we now parse the seconds, including the fractional
  part into a double rather than 2 ints.  This preserves the
  magnitude and precision of the fractional part.  When we print
  it, we now print it as a "%09.6lf" which correctly represents the
  input.

To be honest, why we parse the string timestamp into components,
test the components, then print the components back into a string
timestamp is beyond me.  We should use parse it, test it, then if
it passes, use the original string representation in the database
call.  Maybe someone thought that some implementations wouldn't
take a partial timestamp string like "2017-05-06" and decided to
always produce a full timestamp string even if an abbreviated one
was supplied.  Anyway, I'm leaving it as it is.

ASTERISK-25032 #close
Reported-by: Etienne Lessard

Change-Id: Id407e6221f79a5c1120e1a70bc7e893bbcaf1938

In all non-pbx modules, AST_MODULE_LOAD_FAILURE has been changed
to AST_MODULE_LOAD_DECLINE.  This prevents asterisk from exiting
if a module can't be loaded.  If the user wishes to retain the
FAILURE behavior for a specific module, they can use the "require"
or "preload-require" keyword in modules.conf.

A new API was added to logger: ast_is_logger_initialized().  This
allows asterisk.c/check_init() to print to the error log once the
logger subsystem is ready instead of just to stdout.  If something
does fail before the logger is initialized, we now print to stderr
instead of stdout.

Change-Id: I5f4b50623d9b5a6cb7c5624a8c5c1274c13b2b25

Implement the same buffer size checking done in cel_pgsql.

ASTERISK-26896 #close
Reported by: twisted

Change-Id: Iaacfa1f1de7cb1e9414d121850d2d8c2888f3f48

PQEscapeStringConn() expects the buffer passed in to be an
adequitely sized buffer to write out the escaped SQL value string
into.  It is possible, for large values (such as large values to
Dial with a lot of devices) to have more than our 512+1 byte
allocation and thus cause libpq to create a buffer overrun.

glibc will nicely ABRT asterisk for you, citing a stack smash.

Let's only allocate it to be as large as needed:
If we have a value, then (strlen(value) * 2) + 1 (as recommended
by libpq), and if we have none, just one byte to hold our null
will do.

ASTERISK-26896 #close

Change-Id: If611c734292618ed68dde17816d09dd16667dea2

Radcli is yet another RADIUS client library, generally compatible with
freeradius and radiusclient-ng.

This commit adds autoconf option for detecting it as well and changes
cdr_radius and cel_radius to use its header file in that case.

ASTERISK-26540 #close

Change-Id: I271f0715406334874865ffbce0b354b3a2ca148f

ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.

Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename

This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled.  This variable was only used in lock.c so it
is now initialized in that file only.

ASTERISK-26480 #close

Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966

- Call "rc_openlog" optional. If you do not call,
you will simply NULL instead of a name.

- On the one PID can be only one syslog channel.
And it can already be run in logger.c

- Calling rc_openlog we assigns a new name for
the channel syslog. This unexpected behavior for logger.c.

Most lesser evil, is to agree on a NULL name syslog
if the channel was not launched in logger.c.

It also solves the problem of memory leaks.

ASTERISK-26455 #close

Change-Id: Ic17c38de67583e971d78fe18807d1a9faf8f0afd

Change-Id: Ic7a1236eba2408090fdabb5f717b5fa455ead715

The stringfields refactor to allow adding stringfields to the end of a
structure (f6f4cf45) exposed some
incomplete cleanup code by some stringfield users.

The most noticeable leaker is the logging system where there is a leak for
every log message generated.

ASTERISK-26078 #close
Reported by:  Etienne Lessard
Patches:
      jira_asterisk_26078_v13.patch (license #5621) patch uploaded
      by Richard Mudgett

Change-Id: If6a08b31336b492c3de6f9dfd07c447f8d5a8782

The macro ADD_VENDOR_CODE defined in the cel_radius.c should use the parameter
y not the address of y.

I capture the radius UDP packet via tcpdump, and the AV pairs are not correct,
then i review the source code and compare it with cdr/cdr_radius.c. Fix it and
 it works.

ASTERISK-25647 #close
Reported by: Aaron An
Tested by: Aaron An

Change-Id: I72889bccd8fde120d47aa659edc0e7e6d4d019f0

Remove repeated code on macro of assigned buffer to SQL vars

Change-Id: Icb19ad013124498e172ea1d0b29ccd0ed17deef0

Use a separator string between column names and values for SQL sentences
instead of evaluating the separator to use each time.

This change adds a space after the comma in constructing SQL sentences.
Before the SQL was created like "INSERT INTO cdr(calldate,clid,dst"
without spaces between column name and values.

The files applied this change are cdr/cdr_adaptive_odbc.c, cdr/cdr_pgsql.c,
cel/cel_odbc.c

ASTERISK-25109 #close
Reported By: Rodrigo Ramírez Norambuena <decipher.hk@gmail.com>

Change-Id: Ia5a1a161f5e26e1643703b30f8cc9cf0860cc7ea

Change-Id: Icf88f9f861c6b2a16e5f626ff25795218a6f2723

When appending a column name to the sql buffer, the predicate, "if first is
non-null, use empty string; else, use comma", is identical to the 'SEP' macro
definition. Since they are the same, this patch replaces the redundant
predicate statement with the 'SEP' macro.

Change-Id: Ib8b6138b06a48381723108a05ab8752cb8700509

The code previously used a fixed size of 512 for the SQL
queries. Depending on the size this may require it to grow.

This change makes it so if the buffer size does grow the size
is stored and next time the buffer will be large enough.

Change-Id: I55385899f1c06dee47e4274c2d21538037b2d895

Add feature to set optional schema parameter on configuration file via
'schema' setting.

Fix query to get columns from table while considering schema. If in
the database there exists two tables with same name in distinct schemas
it will return an error when inserting record.

ASTERISK-24967 #close

Change-Id: I691fd2cbc277fcba10e615f5884f8de5d8152f2c

The LOG_ERROR has reference to CDR instead of CEL  for LENGTHEN_BUF1 and
LENGTHEN_BUF2.

ASTERISK-24965 #close
Reported by: Rodrigo Ramirez Norambuena

Change-Id: Icc818697d7d66d34bfe3048cdd15ca2b06c89744

Git does not support the ability to replace a token with a version
string during check-in. While it does have support for replacing a
token on clone, this is somewhat sub-optimal: the token is replaced
with the object hash, which is not particularly easy for human
consumption. What's more, in practice, the source file version was often
not terribly useful. Generally, when triaging bugs, the overall version
of Asterisk is far more useful than an individual SVN version of a file. As a
result, this patch removes Asterisk's support for showing source file
versions.

Specifically, it does the following:

* Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and
  remove passing the version in with the macro. Other facilities
  than 'core show file version' make use of the file names, such as
  setting a debug level only on a specific file. As such, the act of
  registering source files with the Asterisk core still has use. The
  macro rename now reflects the new macro purpose.

* main/asterisk:
  - Refactor the file_version structure to reflect that it no longer
    tracks a version field.
  - Remove the "core show file version" CLI command. Without the file
    version, it is no longer useful.
  - Remove the ast_file_version_find function. The file version is no
    longer tracked.
  - Rename ast_register_file_version/ast_unregister_file_version to
    ast_register_file/ast_unregister_file, respectively.

* main/manager: Remove value from the Version key of the ModuleCheck
  Action. The actual key itself has not been removed, as doing so would
  absolutely constitute a backwards incompatible change. However, since
  the file version is no longer tracked, there is no need to attempt to
  include it in the Version key.

* UPGRADE: Add notes for:
  - Modification to the ModuleCheck AMI Action
  - Removal of the "core show file version" CLI command

Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e

This patch adds a new option to cel_pgsl, "usegmtime", which causes timestamps
to be logged in GMT.

Review: https://reviewboard.asterisk.org/r/4571/

ASTERISK-23186 #close
Reported by: Rodrigo Ramirez Norambuena
patches:
  cel_pgsql.c_add_usegmtime2.patch submitted by Rodrigo Ramirez Norambuena (License 6577)



git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@434284 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch fixes the -Wunused-value -Wunused-variable -Wunused-const-variable
errors caught by clang. Specifically:

* apps/app_queue.c: removed unused qpm_cmd_usage[], qum_cmd_usage[],
                    qsmp_cmd_usage[]
* cel/cel_sqlite3_custom.c: removed unused name[] = "cel_sqlite3_custom"
* channels/chan_pjsip.c: removed unused desc[] = "PJSIP Channel"
* codecs/gsm/src/gsm_create.c: removed unused ident[] = "$Header$"
* funcs/func_env.c:729: Fixed ast_str_append_substr.
* main/editline/np/strlcat.c: removed unused rcsid variable
* main/editline/np/strlcpy.c: removed unused rcsid variable
* main/security_events.c: removed unused TIMESTAMP_STR_LEN
* utils/conf2ael.c: removed unused cfextension_states
* utils/extconf.c: removed unused cfextension_states

Review: https://reviewboard.asterisk.org/r/4526

ASTERISK-24917
Reported by: dkdegroot
patches:
  rb4526.patch submitted by dkdegroot (License 6600)
........

Merged revisions 433693 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 433694 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433695 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch adds AMI event documentation for the Cdr and CEL AMI events.

Note that while these events do share fields with each other and with other
channel related events, they do not contain all of the fields in a standard
channel snapshot, nor is the description of the fields identical. As such,
the patch opts for documentation for each field, for each event.

Review: https://reviewboard.asterisk.org/r/4350/

ASTERISK-24671 #close
Reported by: Dan Jenkins
........

Merged revisions 430862 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@430863 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch adds microsecond precision when inserting a CEL record into a table
with an "eventtime" column of type timestamp, instead of second precision. The
documentation (configs/cel_odbc.conf.sample) was already saying that the
eventtime column included microseconds precision, but that was not the case.

Also, without this patch, if you had a table with an "eventtime" column of
type varchar, you had millisecond precision. With this patch, you also get
microsecond precision in this case.

Review: https://reviewboard.asterisk.org/r/3980

ASTERISK-24283 #close
Reported by: Etienne Lessard
patches:
  cel_odbc_time_precision.patch uploaded by Etienne Lessard (License 6394)
........

Merged revisions 427952 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 427953 from http://svn.asterisk.org/svn/asterisk/branches/12
........

Merged revisions 427954 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@428010 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This gets rid of most old libc free/malloc/realloc and replaces them
with ast_free and friends. When compiling with MALLOC_DEBUG you'll
notice it when you're mistakenly using one of the libc variants. For
the legacy cases you can define WRAP_LIBC_MALLOC before including
asterisk.h.

Even better would be if the errors were also enabled when compiling
without MALLOC_DEBUG, but that's a slightly more invasive header
file change.

Those compiling addons/format_mp3 will need to rerun
./contrib/scripts/get_mp3_source.sh.

ASTERISK-24348 #related
Review: https://reviewboard.asterisk.org/r/4015/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@423978 65c4cc65-6c06-0410-ace0-fbb531ad65f3

ASTERISK-23919 #close
Reported by Malcolm Davenport

Review: https://reviewboard.asterisk.org/r/3802



git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@419592 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch adds support for the PostgreSQL application_name connection setting.
When the appropriate PostgreSQL module's configuration is set with an
application name, the name will be passed to PostgreSQL on connection and
displayed in the database's pg_stat_activity view, as well as in CSV logs. This
aids in managing which applications/servers are connected to a PostgreSQL
database, as well as tracing the activity of those connections.

Review: https://reviewboard.asterisk.org/r/3591

ASTERISK-23737 #close
Reported by: Gergely Domodi
patches:
  pgsql_application_name.patch uploaded by Gergely Domodi (License 6610)


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@418755 65c4cc65-6c06-0410-ace0-fbb531ad65f3

We have faced situation when using CDR and CEL by sqlite3 modules. With system having high load (~100 concurrent calls created by sipp) we found many cdr and cel records missed. There is special finction in sqlite3, that make able to fix this situation - sqlite3_wait_timeout, that also can replace awful code cdr_sqlite3 ad cel_sqlite3 modules. Also this function can be used for aastdb and res_config_sqlite3 to avoid missed writes to sqlite db.

#ASTERISK-23766 #close
Reported by: Igor Goncharovsky

Review: https://reviewboard.asterisk.org/r/3559/
........

Merged revisions 416336 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........

Merged revisions 416337 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 416338 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@416339 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch addresses some aesthetic issues in Asterisk. These are all just
minor tweaks to improve the look of the CLI when used in a variety of
settings. Specifically:
 * A number of chatty verbose messages were removed or demoted to DEBUG
   messages. Verbose messages with a verbosity level of 5 or higher were -
   if kept as verbose messages - demoted to level 4. Several messages
   that were emitted at verbose level 3 were demoted to 4, as announcement
   of dialplan applications being executed occur at level 3 (and so the
   effects of those applications should generally be less).
 * Some verbose messages that only appear when their respective 'debug'
   options are enabled were bumped up to always be displayed.
 * Prefix/timestamping of verbose messages were moved to the verboser
   handlers. This was done to prevent duplication of prefixes when the
   timestamp option (-T) is used with the CLI.
 * Verbose magic is removed from messages before being emitted to
   non-verboser handlers. This prevents the magic in multi-line verbose
   messages (such as SIP debug traces or the output of DumpChan) from
   being written to files.
 * _Slightly_ better support for the "light background" option (-W) was
   added. This includes using ast_term_quit in the output of XML
   documentation help, as well as changing the "Asterisk Ready" prompt to
   bright green on the default background (which stands a better chance of
   being displayed properly than bright white).

Review: https://reviewboard.asterisk.org/r/3547/



git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@414798 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This resolves a large number of compiler warnings from GCC 4.10 which
cause the build to fail under dev mode. The vast majority are
signed/unsigned mismatches in printf-style format strings.
........

Merged revisions 413586 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........

Merged revisions 413587 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 413588 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@413589 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Asterisk's RADIUS module currently build against libradiusclient-ng, but this
project has been superseeded by libfreeradius-client. The API is 99% compatible
except that the header name has changed, the library name has changed, and
the configuration file location has changed.

(closes issue ASTERISK-22980)
Reported by: Jeremy Lainé
Patches:
     freeradius-client.patch uploaded by sharky (license 6561)
........

Merged revisions 406801 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........

Merged revisions 406802 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 406803 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@406825 65c4cc65-6c06-0410-ace0-fbb531ad65f3

The cel_manager module did not properly handle the case where the
configuration file was invalid. The module will now output a warning
message and disable itself if this occurs.

Reported by: Bryan Walters
........

Merged revisions 405581 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........

Merged revisions 405582 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 405583 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@405584 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Upon reload the module unconditionally "unloaded" the module (freeing memory
and setting pointers to NULL) and then when attempting a "load" if the config
file had not changed then nothing would be reinitialized.

By moving the "unload" to occur conditionally (reload only) after an attempted
configuration load, but before module "loading" alleviates the issue. The module
now loads/unloads/reloads correctly.

(closes issue ASTERISK-22871)
Reported by: Matteo
........

Merged revisions 404857 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........

Merged revisions 404858 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 404859 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@404860 65c4cc65-6c06-0410-ace0-fbb531ad65f3

A deadlock can happen between a thread unloading or reloading the cel_pgsql
module and the core_event_dispatcher taskprocessor thread. Description of
what is happening:

Thread 1 (for example, a netconsole thread):

    a "module reload cel_pgsql" is launched
    the thread enter the "my_unload_module" function (cel_pgsql.c)
    the thread acquire the write lock on psql_columns
    the thread enter the "ast_event_unsubscribe" function (event.c)
    the thread try to acquire the write lock on ast_event_subs[sub->type]

Thread 2 (core_event_dispatcher taskprocessor thread):

    the taskprocessor pop a CEL event
    the thread enter the "handle_event" function (event.c)
    the thread acquire the read lock on ast_event_subs[sub->type]
    the thread callback the "pgsql_log" function (cel_pgsql.c), since it's a subscriber of CEL events
    the thread try to acquire a read lock on psql_columns

(closes issue ASTERISK-22854)
Reported by: Etienne Lessard
Patches:
     cel_pgsql_fix_deadlock_event.patch uploaded by hexanol (license 6394)
........

Merged revisions 404603 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........

Merged revisions 404604 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 404605 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@404606 65c4cc65-6c06-0410-ace0-fbb531ad65f3