When appending all formats of a type all the codecs are iterated
and added. This operation was incorrectly adding the ast_format_none
format which is special in that it is supposed to be used when no
format is present. It shouldn't be appended.

ASTERISK-25535

Change-Id: I7b00f3bdf4a5f3022e483d6ece602b1e8b12827c

This change adds handling of dead worker threads when moving them
to be active. When this happens the worker thread is removed from
both the active and idle threads container. If no threads are able
to be moved to active then the pool grows as configured.

A unit test has also been added which thrashes the idle timeout
and thread activation to exploit any race conditions between the
two.

ASTERISK-25546 #close

Change-Id: I6c455f9a40de60d9e86458d447b548fb52ba1143

ASTERISK-25533 #close

Change-Id: Ie1a9d1a6511b3f1a56b93d04475fbf8a4e40010a

Some codecs that may be a third party library to Asterisk need to have
knowledge of the format attributes that were negotiated. Unfortunately,
when the great format migration of Asterisk 13 occurred, that ability
was lost.

This patch adds an API call, ast_format_attribute_get, to the core
format API, along with updates to the unit test to check the new API
call. A new callback is also now available for format attribute modules,
such that they can provide the format attribute values they manage.

Note that the API returns a void *. This is done as the format attribute
modules themselves may store format attributes in any particular manner
they like. Care should be taken by consumers of the API to check the
return value before casting and dereferencing. Consumers will obviously
need to have a priori knowledge of the type of the format attribute as
well.

Change-Id: Ieec76883dfb46ecd7aff3dc81a52c81f4dc1b9e3

clock_gettime() is, unfortunately, not portable. But I did like that
over our usual `ts.tv_nsec = tv.tv_usec * 1000` copy/paste code we
usually do when we want a timespec and all we have is ast_tvnow().

This patch adds ast_tsnow(), which mimics ast_tvnow(), but returns a
timespec. If clock_gettime() is available, it will use that. Otherwise
ast_tsnow() falls back to using ast_tvnow().

Change-Id: Ibb1ee67ccf4826b9b76d5a5eb62e90b29b6c456e

A testsuite test recently failed due to a crash that occurred in the DNS
core. The problem was that the test could not resolve an address, did
not set a result on the DNS query, and then indicated the query was
completed. The DNS core does not handle the case of a query with no
result gracefully, and so there is a crash.

This changeset makes the DNS system resolver set a result with a
zero-length answer in the case that a DNS resolution failure occurs
early. The DNS core now also will accept such a response without
treating it as invalid input. A unit test was updated to no longer treat
setting a zero-length response as off-nominal.

Change-Id: Ie56641e22debdaa61459e1c9a042e23b78affbf6

Fixes for issues with the ASTERISK-24934 patch.

* Fixed ast_escape_alloc() and ast_escape_c_alloc() if the s parameter is
an empty string.  If it were an empty string the functions returned NULL
as if there were a memory allocation failure.  This failure caused the AMI
VarSet event to not get posted if the new value was an empty string.

* Fixed dest buffer overwrite potential in ast_escape() and
ast_escape_c().  If the dest buffer size is smaller than the space needed
by the escaped s parameter string then the dest buffer would be written
beyond the end by the nul string terminator.  The num parameter was really
the dest buffer size parameter so I renamed it to size.

* Made nul terminate the dest buffer if the source string parameter s was
an empty string in ast_escape() and ast_escape_c().

* Updated ast_escape() and ast_escape_c() doxygen function description
comments to reflect reality.

* Added some more unit test cases to /main/strings/escape to cover the
empty source string issues.

ASTERISK-25255 #close
Reported by: Richard Mudgett

Change-Id: Id77fc704600ebcce81615c1200296f74de254104

This patch adds a new API to the Asterisk core that acts as a media
cache. The core API itself is mostly a thin wrapper around some bucket
API provided implementation that itself acts as the mechanism of
retrieval for media. The media cache API in the core provides the
following:
 * A very thin in-memory cache of the active bucket_file items. Unlike a
   more traditional cache, it provides no expiration mechanisms. Most
   queries that hit the in-memory cache will also call into the bucket
   implementations as well. The bucket implementations are responsible
   for determining whether or not the active record is active and valid.
   This makes sense for the most likely implementation of a media cache
   backend, i.e., HTTP. The HTTP layer itself is the actual arbiter of
   whether or not a record is truly active; as such, the in-memory cache
   in the core has to defer to it.
 * The ability to create new items in the media cache from local
   resources. This allows for re-creation of items in the cache on
   restart.
 * Synchronization of items in the media cache to the AstDB. This
   also includes various pieces of important metadata.

The API provides sufficient access that higher level APIs, such as the
file or app APIs, do not have to worry about the semantics of the bucket
APIs when needing to playback a resource.

In addition, this patch provides unit tests for the media cache API. The
unit tests use a fake bucket backend to verify correctness.

Change-Id: I11227abbf14d8929eeb140ddd101dd5c3820391e

This patch adds more tests that exercise the device state API. This includes:

* Tests that cover adding a device state provider, as well as deleting a
  device state provider. This also verifies that you cannot add an
  already added device state provider, and cannot delete an already
  deleted device state provider.
* A test that covers changing device state and receiving said updates
  from a device state subscriber. This also covers hitting both the
  device state cache as well as a custom device state provider.
* A test that covers converting device state to channel state and device
  state values to a string representation and back.
* A test that covers obtaining device state from an active channel and a
  channel driver that provides its own device state.

Change-Id: I2adca67ffb405cd8625a5d6df1e3f9b3d945c08d

This patch enhances the bucket API in two ways.

First, since ast_bucket and ast_bucket_file instances are immutable, a 'clone'
operation has been added that provides a 'clone' of an existing
ast_bucket/ast_bucket_file object. Note that this makes use of the
ast_sorcery_copy operation, along with the copy callback handler on the
"bucket" and "file" object types for the bucket sorcery instance.

Second, there is a need for the bucket API to ask a wizard if an object
is stale. This is particularly useful with the upcoming media cache
enhancements, where we want to ask the backing data storage if the
object we are currently operating on has known updates. This patch adds
API calls for ast_bucket and ast_bucket_file objects, which callback
into their respective sorcery wizards via the sorcery API.

Unit tests have also been added to cover the respective
ast_bucket/ast_bucket_file clone and staleness operations.

Change-Id: Ib0240ba915ece313f1678a085a716021d75d6b4a

This patch enhances the sorcery API to allow for sorcery wizards to
determine if an object is stale. This includes the following:

* Sorcery objects now have a timestamp that is set on creation. Since
  sorcery objects are immutable, this can be used by sorcery wizards to
  determine if an object is stale.

* A new API call has been added, ast_sorcery_is_stale. This API call
  queries the wizards associated with the object, calling a new callback
  function 'is_stale'. Note that if a wizard does not support the new
  callback, objects are always assumed to not be stale.

* Unit tests have been added that cover the new API call.

Change-Id: Ica93c6a4e8a06c0376ea43e00cf702920b806064

Analyzing the code shows that the unit test summary and description
strings should not end with a new-line character.  Where these strings are
used in the code a new-line is provided for output.

Change-Id: I2f4f37988ec363c8d1c5077a2fc8ca841c5cd30c

Analyzing the code shows that the unit test summary and description
strings should not end with a new-line character.  Where these strings are
used in the code a new-line is provided for output.

Change-Id: I129284f5e7ca93d82532334076da4c462d3d9fba

Change-Id: Icf5f13c8e1c2c92a4473bb573ed2dd856ce1b64e

* Fix query_set destruction before we are done kicking the queries off.

* Fixed no queries requested handling.

* Add empty queries request unit test.

* Added missing allocation check in ast_dns_query_set_add().

* Made initial pjsip resolving query vector slightly larger.

ASTERISK-25115
Reported by: John Bigelow

Change-Id: Ie8be8347d0992e93946d72b6e7b1299727b038f2

Those trailing newlines mess up test formatting.

Change-Id: I5e3f3a55b82c9d7acb9661201d4993d1958f1185

Although ast_context_find, ast_context_find_or_create and
ast_context_destroy perform locking of the contexts table,
any context pointer can become invalid at any time that the
contexts table is unlocked. This change adds locking around
all complete operations involving these functions.

Places where ast_context_find was followed by ast_context_destroy
have been replaced with calls ast_context_destroy_by_name.

ASTERISK-25094 #close
Reported by: Corey Farrell

Change-Id: I1866b6787730c9c4f3f836b6133ffe9c820734fa

So this issue is a bit complicated. Since it is possible to pass values to AMI
that contain a '\r\n' (or other similar sequences) these values need to be
escaped. One way to solve this is to escape the values and then pass the escaped
values to the AMI variable parameter string building function. However, this
puts the onus on the pre-build function to escape all string values. This
potentially requires a fair amount of changes along with a lot of string
allocations/freeing for all values.

Surely there is a way to push this complexity down a level into the string
building function itself? This of course is possible, but ends up requiring a
way to distinguish between strings that need to be escaped and those that don't.
The best way to handle this is by introducing a new format specifier in the
format string. For instance a %s (no escape) and %S (escape). However, that is
a bit weird and unexpected.

So faced with those possibilities this patch implements a limited version of the
first option. Instead of attempting to escape all string values this patch only
escapes those values that make sense. This approach limits the number of changes
and doesn't suffer from the odd format specifier problem.

ASTERISK-24934 #close
Reported by: warren smith

Change-Id: Ib55a5b84fe0481b0f2caaaab68c566f392c0aac0

This change adds a CLI command which can perform memory cache thrashing as well
as unit tests which perform thrashing under the following configurations:

1. Low number of unique objects that go stale after 1 second
2. Low number of unique objects that expire after 1 second
3. Low number of unique objects which are constantly updated
4. Large number of unique objects which exceed a defined cache size
5. Large number of unique objects which exceed a defined cache size
   that also expire and go stale rapidly
6. Large number of unique objects which expire and go stale rapidly
7. Large number of unique objects

For all of the above there are a large number of threads constantly
attempting to retrieve random objects and each test runs for a few
seconds.

ASTERISK-25067
Reported by: Matt Jordan

Change-Id: I8c8ceff977332c80ed4a31f10d694d48552b2f78

This patch fixes a number of errors and warning messages in the doxygen
log. Specifically, it addresses:
* A number of files incorrectly places a '\brief' tag immediately after
  a '\file' tag. Doing so emits a warning, as '\file' takes an optional
  argument specifying which file the doxygen comment is for. As '\brief'
  is not a file, doxygen was unamused.
* A grouping of Stasis Topics and Messages in rtp_engine.h was
  incorrectly terminated. We now correctly terminate the grouping, which
  prevents members of rtp_engine.h from showing up in the wrong group.
* Group indicators which are not part of the Stasis Topics and Messages
  group were removed. Group indicators without an \addtogroup or
  \ingroup have no meaning.

Change-Id: Ia1415ffec6767e27233ae1cae5ed5970de5656d4

The config wizard was always pulling the first occurrence of
a variable from an ast_variable list but this gets the template
value from the list instead of any overridden value.  This patch
creates ast_variable_find_last_in_list() in config.c and updates
res_pjsip_config_wizard to use it instead of
ast_variable_find_in_list.  Now the overridden values, where they
exist, are used instead of template variables.

Updated test_config to test the new API.

ASTERISK-25089 #close

Reported-by: George Joseph <george.joseph@fairview5.com>
Tested-by: George Joseph <george.joseph@fairview5.com>
Change-Id: Ifa7ddefc956a463923ee6839dd1ebe021c299de4

These modules save a pointer to the context they create on load, and
use that pointer to destroy the context at unload.  It is not safe
to save this pointer, it is replaced during load of pbx_config,
pbx_lua or pbx_ael.

This change causes the modules to pass NULL to ast_context_destroy,
a safer way to perform the unregistration since it does not use
a pointer that could become invalid.

ASTERISK-25085 #close
Reported by: Corey Farrell

Change-Id: I6a00ec8e38046058f97dc703e1adcde9bf517835

Change-Id: Icf88f9f861c6b2a16e5f626ff25795218a6f2723

Currently you can 'apply' a wizard to an object type but the wizard
always goes at the end of the object type's wizard list.  This patch
adds a new ast_sorcery_insert_wizard_mapping function that allows
you to insert a wizard anyplace in the list.  I.E.  You could
add a caching wizard to an object type and place it before all
wizards.

ast_sorcery_get_wizard_mapping_count and
ast_sorcery_get_wizard_mapping were added to allow examination
of the mapping list.

ast_sorcery_remove_mapping was added to remove a mapping by name.

As part of this patch, the object type's wizard list was converted
from an ao2_container to an AST_VECTOR_RW.

A new test was added to test_sorcery for this capability.

ASTERISK-25044 #close

Change-Id: I9d2469a9296b2698082c0989e25e6848dc403b57

Based on feedback from Corey Farrell and Y Ateya, a few new
macros have been added...

AST_VECTOR_REMOVE which takes a parameter to indicate if
order should be preserved.

AST_VECTOR_ADD_SORTED which adds an element to
a sorted vector.

AST_VECTOR_RESET which cleans all elements from the vector
leaving the storage intact.

Change-Id: I41d32dbdf7137e0557134efeff9f9f1064b58d14

After using the new vector stuff for real I found...

A bug in AST_VECTOR_INSERT_AT that could cause a seg fault.

The callbacks needed to be closer to ao2_callback in behavior
WRT to CMP_MATCH and CMP_STOP behavior and the ability to return
a vector of matched entries.

A pre-existing issue with APPEND and REPLACE was also fixed.

I also added a new macro to test.h that acts like ast_test_validate
but also accepts a return code variable and a cleanup label.  As well
as printing the error, it sets the rc variable to AST_TEST_FAIL and
does a goto to the specified label on error.  I had a local version
of this in test_vector so I just moved it.

ASTERISK-25045

Change-Id: I05e5e47fd02f61964be13b7e8942bab5d61b29cc

Renamed AST_VECTOR_INSERT to AST_VECTOR_REPLACE because it really
does replace not insert.  The few users of AST_VECTOR_INSERT were
refactored.  Because these are macros, there should be no ABI
compatibility issues.

Added AST_VECTOR_INSERT_AT that actually inserts an element into the
vector at a specific index pushing existing elements to the right.

Added AST_VECTOR_GET_CMP that can retrieve from the vector based
on a user-provided compare function.

Added AST_VECTOR_CALLBACK function that will execute a function
for each element in the vector.  Similar to ao2_callback and
ao2_callback_data functions although the vector callback can take
a variable number of arguments.  This should allow easy migration
to a vector where a container might be too heavy.

Added read/write locked vector and lock manipulation macros.

Added unit tests.

ASTERISK-25045 #close

Change-Id: I2e07ecc709d2f5f91bcab8904e5e9340609b00e0

This function allows code to run ao2_ref against the real
object associated with a weakproxy.  It is useful when
all of the following conditions are true:
* You have a pointer to weakproxy.
* You do not have or need a pointer to the real object.
* You need to ensure the real object exists and is not
  destroyed during a process.

In this case it's wasteful to store a pointer to the real
object just for the sake of releasing it later.

Change-Id: I38a319b83314de75be74207a8771aab269bcca46

The query set documentation states that upon completion queries can be
retrieved for the lifetime of the query set. This is a reasonable
expectation but does not currently occur. This was originally done
to resolve a circular reference between queries and query sets, but
in practice the query can be kept.

This change makes it so a query does not have a reference to the
query set until it begins resolving. It also makes it so that the
reference is given up upon the query being completed. This allows
the queries to remain for the lifetime of the query set. As the
query set on the query is only useful to the query set functionality
and only for the lifetime that the query is resolving this is safe
to do.

ASTERISK-24994 #close
Reported by: Joshua Colp

Change-Id: I54e09c0cb45475896654e7835394524e816d1aa0

This change adds the following:

1. A query set implementation. This is an API that allows queries to be executed in parallel and once all have completed a callback is invoked.
2. Unit tests for the query set implementation.
3. An external PJSIP resolver which uses the DNS core API to do NAPTR, SRV, AAAA, and A lookups.

For the resolver it will do NAPTR, SRV, and AAAA/A lookups in parallel. If NAPTR or SRV
are available it will then do more queries. And so on. Preference is NAPTR > SRV > AAAA/A,
with IPv6 preferred over IPv4. For transport it will prefer TLS > TCP > UDP if no explicit
transport has been provided. Configured transports on the system are taken into account to
eliminate resolved addresses which have no hope of completing.

ASTERISK-24947 #close
Reported by: Joshua Colp

Change-Id: I56cb03ce4f9d3d600776f36928e0b3e379b5d71e

Update test_astobj2_weaken to use the new AST_REGISTER_FILE macro.

Change-Id: Ieedadf16610f2e042f393e0501a36447cd07f83d

This implements "weak" references.  The weakproxy object is a real ao2 with
normal reference counting of its own.  When a weakproxy is pointed to a normal
object they hold references to each other.  The normal object is automatically
freed when a single reference remains (the weakproxy).  The weakproxy also
supports subscriptions that will notify callbacks when it does not point
to any real object.

ASTERISK-24936 #close
Reported by: Corey Farrell

Change-Id: Ib9f73c02262488d314d9d9d62f58165b9ec43c67

Git does not support the ability to replace a token with a version
string during check-in. While it does have support for replacing a
token on clone, this is somewhat sub-optimal: the token is replaced
with the object hash, which is not particularly easy for human
consumption. What's more, in practice, the source file version was often
not terribly useful. Generally, when triaging bugs, the overall version
of Asterisk is far more useful than an individual SVN version of a file. As a
result, this patch removes Asterisk's support for showing source file
versions.

Specifically, it does the following:

* Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and
  remove passing the version in with the macro. Other facilities
  than 'core show file version' make use of the file names, such as
  setting a debug level only on a specific file. As such, the act of
  registering source files with the Asterisk core still has use. The
  macro rename now reflects the new macro purpose.

* main/asterisk:
  - Refactor the file_version structure to reflect that it no longer
    tracks a version field.
  - Remove the "core show file version" CLI command. Without the file
    version, it is no longer useful.
  - Remove the ast_file_version_find function. The file version is no
    longer tracked.
  - Rename ast_register_file_version/ast_unregister_file_version to
    ast_register_file/ast_unregister_file, respectively.

* main/manager: Remove value from the Version key of the ModuleCheck
  Action. The actual key itself has not been removed, as doing so would
  absolutely constitute a backwards incompatible change. However, since
  the file version is no longer tracked, there is no need to attempt to
  include it in the Version key.

* UPGRADE: Add notes for:
  - Modification to the ModuleCheck AMI Action
  - Removal of the "core show file version" CLI command

Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e

This patch fixes a variety of clang compiler warnings for unit tests. This
includes autological comparison issues, ignored return values, and
interestingly enough, one embedded function. Fun!

Review: https://reviewboard.asterisk.org/r/4555

ASTERISK-24917
Reported by: dkdegroot
patches:
  rb4555.patch submitted by dkdegroot (License 6600)
........

Merged revisions 434705 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 434706 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@434707 65c4cc65-6c06-0410-ace0-fbb531ad65f3

The NAPTR and SRV branches were worked on independently and
resulted in some code being duplicated in each. Since both
have been merged into trunk now, this patch reduces the
duplication by factoring out common code into its own
source files.



git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@434490 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@434170 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@434154 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This change adds support for parsing SRV records and consuming their values
in an easy fashion. It also adds automatic sorting of SRV records according
to RFC 2782.

Tests have also been included which cover parsing, sorting, and off-nominal
cases where the record is corrupted.

ASTERISK-24931 #close
Reported by: Joshua Colp

Review: https://reviewboard.asterisk.org/r/4528/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433889 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This change adds an abstracted core DNS API which resembles the API described
here[1]. The API provides a pluggable mechanism for resolvers and also a
consistent view for records. Both synchronous and asynchronous queries are
supported.

This change also adds a res_resolver_unbound module which uses the libunbound
library to provide resolution.

Unit tests have also been written for all of the above to confirm the API and
functionality.

ASTERISK-24834 #close
Reported by: Matt Jordan

ASTERISK-24836 #close
Reported by: Matt Jordan

Review: https://reviewboard.asterisk.org/r/4474/
Review: https://reviewboard.asterisk.org/r/4512/

[1] https://wiki.asterisk.org/wiki/display/AST/Asterisk+DNS+API


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433370 65c4cc65-6c06-0410-ace0-fbb531ad65f3

When r432935 was merged, it did correctly fix a situation where a FILE read
operation on the middle of a file buffer would not read the requested length
in the parameters passed to the FILE function. Unfortunately, it would also
allow the FILE function to append more bytes than what was available in the
buffer if the length exceeded the end of the buffer length.

This patch takes the minimum of the remaining bytes in the buffer along with
the calculated length to append provided by the original patch, and uses
that as the length to append in the return result. This patch also updates
the unit tests with the scenarios that were originally pointed out in
ASTERISK-21765 that the original implementation treated incorrectly.

ASTERISK-21765
........

Merged revisions 433173 from http://svn.asterisk.org/svn/asterisk/branches/11
........

Merged revisions 433174 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433175 65c4cc65-6c06-0410-ace0-fbb531ad65f3