git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396857 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Change r395954 reordered some stasis object destruction, which should
have been fine. Unfortunately, it caused some hard to reproduce issues
related to objects being accessed after they had been destroyed. The
patch in r396329 fixed the destruction order problem; this patch
addresses the underlying issue. A few other stasis-related fixes were
also added.

 * Add ref-bumps around areas where objects may get transitively
   destroyed. (For example, where we lock a topic, unref a subscription,
   which unrefs the topic, which explodes the topic when we try to
   unlock it.)

 * Wrote an extensive doxygen page about Stasis implementation,
   relationships between objects, lifecycles of objects, how the
   refcounting works, etc. Many other comments were added, corrected, or
   cleaned up.

 * Added an assert to the topic dtor to catch extra ref decrements.

 * Fixed type used after destruction errors for graceful shutdown in
   stasis_channels.c.

 * I added two unit tests in an attempt to catch destruction order
   issues. Since the underlying cause is a race condition, though, the
   tests rarely failed even when the code was wrong.

 * Fixed a leak in stasis_cache_pattern.c.

(closes issue ASTERISK-22243)
Review: https://reviewboard.asterisk.org/r/2746/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396842 65c4cc65-6c06-0410-ace0-fbb531ad65f3

........

Merged revisions 396656 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........

Merged revisions 396657 from http://svn.asterisk.org/svn/asterisk/branches/11


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396658 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Depending on certain conditions it was possible for the hashtab counting thread
to starve other threads, preventing them from executing in the expected fashion.
This change adds a sleep to allow the others to do what they need to do. While
this doesn't thrash the hashtab as much as previously, it at least works.

(closes issue ASTERISK-22276)
Reported by: Matt Jordan
........

Merged revisions 396619 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........

Merged revisions 396620 from http://svn.asterisk.org/svn/asterisk/branches/11


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396621 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396560 65c4cc65-6c06-0410-ace0-fbb531ad65f3

The debug logs added in r396528 neglected to account for suseconds_t
being an int.

See r392076 for more info.



git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396559 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Otherwise, it doesn't show up in the automated test failures


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396535 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Disabling DEBUG_THREADS caused this test to fail on the 32-bit build agent.
Adding some debugging to see why it thinks the test is timing out.


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396528 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396480 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396463 65c4cc65-6c06-0410-ace0-fbb531ad65f3

The Stasis changes in r395954 had an unanticipated side effect: messages
published directly to an _all topic does not get forwarded to the
corresponding caching topic.

This patch fixes that by changing how caching topics forward messages,
and how the caching pattern forwards are setup.

For the caching pattern, the all_topic is forwarded to the
all_topic_cached. This forwards messages published directly to the
all_topic to all_topic_cached.

In order to avoid duplicate messages on all_topic_cached, caching topics
were changed to no longer forward uncached messages. Subscribers to an
individual caching topic should only expect to receive cache updates,
and subscription change messages. Since individual caching topics are
new, this shouldn't be a problem.

There are a few minor changes to the pre-cache split behavior.

 * For topics changed to use the caching pattern, the all_topic_cached
   will forward snapshots in addition to cache updates. Since
   subscribers by design ignore unexpected messages, this should be
   fine.

 * Caching topics that don't use the caching pattern no longer forward
   non-cache updates. This makes no difference for the current caching
   topics.

   * mwi_topic_cached, channel_by_name_topic and
     presence_state_topic_cached have no subscribers

   * device_state_topic_cached's only subscriber only processes cache
     udpates

(issue ASTERISK-22243)
Review: https://reviewboard.asterisk.org/r/2738


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396329 65c4cc65-6c06-0410-ace0-fbb531ad65f3

In tracking down some unit tests failures, I ended up reading the fine
print[1] regarding Jansson's thread safety.

In short:
 1. Ref-counting is non-atomic.
 2. json_dumps() and friends are not thread safe.

This patch adds locking where necessary to our ast_json_* wrapper API,
with documentation in json.h describing the thread safety limitations of
the API.

 [1]: http://www.digip.org/jansson/doc/2.4/portability.html#thread-safety

Review: https://reviewboard.asterisk.org/r/2716/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396119 65c4cc65-6c06-0410-ace0-fbb531ad65f3

More missing include files. :-\


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396062 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch does the following:
 * It moves the pickup code out of features.c and into pickup.c
 * It removes the vast majority of dead code out of features.c. In particular,
   this includes the parking code.

(issue ASTERISK-22134)



git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396060 65c4cc65-6c06-0410-ace0-fbb531ad65f3

In working with res_stasis, I discovered a significant limitation to
the current structure of stasis_caching_topics: you cannot subscribe
to cache updates for a single channel/bridge/endpoint/etc.

To address this, this patch splits the cache away from the
stasis_caching_topic, making it a first class object. The stasis_cache
object is shared amongst individual stasis_caching_topics that are
created per channel/endpoint/etc. These are still forwarded to global
whatever_all_cached topics, so their use from most of the code does
not change.

In making these changes, I noticed that we frequently used a similar
pattern for bridges, endpoints and channels:

     single_topic  ---------------->  all_topic
           ^
           |
     single_topic_cached  ----+---->  all_topic_cached
                              |
                              +---->  cache

This pattern was extracted as the 'Stasis Caching Pattern', defined in
stasis_caching_pattern.h. This avoids a lot of duplicate code between
the different domain objects.

Since the cache is now disassociated from its upstream caching topics,
this also necessitated a change to how the 'guaranteed' flag worked
for retrieving from a cache. The code for handling the caching
guarantee was extracted into a 'stasis_topic_wait' function, which
works for any stasis_topic.

(closes issue ASTERISK-22002)
Review: https://reviewboard.asterisk.org/r/2672/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395954 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395881 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395837 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Improve reliability of attended transfer merge and link tests.
Stop using ast_log(LOG_ERROR, ...); in favor of ast_test_status_update
Remove fred and eve channel helpers since they are not necessary


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395653 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This renames all files and API calls from several variants of
Stasis-HTTP to ARI including:
* Stasis-HTTP -> ARI
* STASIS_HTTP -> ARI
* stasis_http -> ari (ast_ari for global symbols, file names as well)
* stasis http -> ARI

Review: https://reviewboard.asterisk.org/r/2706/
(closes issue ASTERISK-22136)


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395603 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395588 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395381 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch renames the bridging* files to bridge*. This may seem pedantic
and silly, but it fits better in line with current Asterisk naming conventions:
* channel is not "channeling"
* monitor is not "monitoring"
etc.

A bridge is an object. It is a first class citizen in Asterisk. "Bridging" is
the act of using a bridge on a set of channels - and the API that fulfills that
role is more than just the action.

(closes issue ASTERISK-22130)


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395378 65c4cc65-6c06-0410-ace0-fbb531ad65f3

The stasis_cache_update messages are somewhat cumbersome to handle
with the stasis_message_router. Since all updates have the same
message type, they are normally handled with the same route.

Since caching itself is a first class component of stasis-core, it
makes sense for the router to handle the cache update messages itself.
This patch adds stasis_message_router_add_cache_update() and
stasis_message_router_remove_cache_update() to handle the routing of
stasis_cache_update messages.

This patch also corrects an issue with manager_{bridging,channels}.c,
where events might be reordered. The reordering occurs because the
components use different message routers, which they needed because
they both needed to route cache update messages. They now both use
manager's router, and add cache routes for just the cache updates they
are interested in.

(closes issue ASTERISK-22038)
Review: https://reviewboard.asterisk.org/r/2677/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395118 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395074 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This adds a new CEL event type, AST_CEL_LOCAL_OPTIMIZE, to represent
local channel optimizations. Local channel optimizations were one of
several things conveyed by the now defunct BRIDGE_UPDATE event type.
This also adds a unit test to test generation of this new CEL event.

Review: https://reviewboard.asterisk.org/r/2676/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@394870 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This adds CEL support for blind and attended transfers and call pickup.
During the course of adding this functionality I noticed that
CONF_ENTER, CONF_EXIT, and BRIDGE_TO_CONF events are particularly
useless without a bridge identifier, so I added that as well.

This adds tests for blind transfers, several types of attended
transfers, and call pickup.

The extra field in CEL records now consists of a JSON blob whose fields
are defined on a per-event basis.

Review: https://reviewboard.asterisk.org/r/2658/
(closes issue ASTERISK-21565)


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@394858 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Since ast_hangup() is effectively a channel destructor, it should be a
void function.

* Make the few silly callers checking the return value no longer do so.
Only the CDR and CEL unit tests checked the return value.

* Make all callers take advantage of the NULL safe change and remove the
NULL check before the call.


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@394623 65c4cc65-6c06-0410-ace0-fbb531ad65f3

The JSON test attempted an overly clever use of RAII_VAR to run code
at the beginning and end of each test, in order to validate that no
JSON objects were leaked during the test.

The problem is that the validation code would run during the initial
load, when the tests were initialized. This happens during startup,
when other parts of the system might actively be allocating and
freeing JSON objects.

This patch changes the RAII_VAR to use the new
ast_test_register_{init,cleanup} functions to run the validations
properly.

(closes issue ASTERISK-21978)
Review: https://reviewboard.asterisk.org/r/2669/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@394203 65c4cc65-6c06-0410-ace0-fbb531ad65f3

When I corrected the CEL test crash in r394037, I didn't quite pay attention
to how the globals and locals were being shuffled around in the cleanup
callback. I removed the nulling of the global variables, which caused them
to be double cleaned.

This patch puts the global nulling code back (since the vars are cleaned up
by RAII_VARs), and removes the explicit ao2_cleanup() (since they were no-ops,
because the variables had just been nulled).



git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@394089 65c4cc65-6c06-0410-ace0-fbb531ad65f3

The voicemail_api test had code like strncmp(a, b, sizeof(a)), but a was a
char pointer, instead of a literal or char array. This meant that sizeof was
the size of the pointer, not the length of the string.

Since the string is in a stringfield and should be null terminated, I just
changed it to a plain strcmp.


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@394050 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@394037 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@393679 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch is the first step in adding recording support to the
Asterisk REST Interface.

Recordings are stored in /var/spool/recording. Since recordings may be
destructive (overwriting existing files), the API rejects attempts to
escape the recording directory (avoiding issues if someone attempts to
record to ../../lib/sounds/greeting, for example).

(closes issue ASTERISK-21594)
(closes issue ASTERISK-21581)
Review: https://reviewboard.asterisk.org/r/2612/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@393550 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch started with the simple idea of changing the /events data
model to be more sane. The original model would send out events like:

    { "stasis_start": { "args": [], "channel": { ... } } }

The event discriminator was the field name instead of being a value in
the object, due to limitations in how Swagger 1.1 could model objects.
While technically sufficient in communicating event information, it was
really difficult to deal with in terms of client side JSON handling.

This patch takes advantage of a proposed extension[1] to Swagger which
allows type variance through the use of a discriminator field. This had
a domino effect that made this a surprisingly large patch.

 [1]: https://groups.google.com/d/msg/wordnik-api/EC3rGajE0os/ey_5dBI_jWcJ

In changing the models, I also had to change the swagger_model.py
processor so it can handle the type discriminator and subtyping. I took
that a big step forward, and using that information to generate an
ari_model module, which can validate a JSON object against the Swagger
model.

The REST and WebSocket generators were changed to take advantage of the
validators. If compiled with AST_DEVMODE enabled, JSON objects that
don't match their corresponding models will not be sent out. For REST
API calls, a 500 Internal Server response is sent. For WebSockets, the
invalid JSON message is replaced with an error message.

Since this took over about half of the job of the existing JSON
generators, and the .to_json virtual function on messages took over the
other half, I reluctantly removed the generators.

The validators turned up all sorts of errors and inconsistencies in our
data models, and the code. These were cleaned up, with checks in the
code generator avoid some of the consistency problems in the future.

 * The model for a channel snapshot was trimmed down to match the
   information sent via AMI. Many of the field being sent were not
   useful in the general case.
 * The model for a bridge snapshot was updated to be more consistent
   with the other ARI models.

Another impact of introducing subtyping was that the swagger-codegen
documentation generator was insufficient (at least until it catches up
with Swagger 1.2). I wanted it to be easier to generate docs for the API
anyways, so I ported the wiki pages to use the Asterisk Swagger
generator. In the process, I was able to clean up many of the model
links, which would occasionally give inconsistent results on the wiki. I
also added error responses to the wiki docs, making the wiki
documentation more complete.

Finally, since Stasis-HTTP will now be named Asterisk REST Interface
(ARI), any new functions and files I created carry the ari_ prefix. I
changed a few stasis_http references to ari where it was non-intrusive
and made sense.

(closes issue ASTERISK-21885)
Review: https://reviewboard.asterisk.org/r/2639/



git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@393529 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch moves the RESTful URL's around to more appropriate
locations for release.

The /stasis URL's are moved to /ari, since Asterisk REST Interface was
a more appropriate name than Stasis-HTTP. (Most of the code still has
stasis_http references, but they will be cleaned up after there are no
more outstanding branches that would have merge conflicts with such a
change).

A larger change was moving the ARI events WebSocket off of the shared
/ws URL to its permanent home on /ari/events. The Swagger code
generator was extended to handle "upgrade: websocket" and
"websocketProtocol:" attributes on an operation.

The WebSocket module was modified to better handle WebSocket servers
that have a single registered protocol handler. If a client
connections does not specify the Sec-WebSocket-Protocol header, and
the server has a single protocol handler registered, the WebSocket
server will go ahead and accept the client for that subprotocol.

(closes issue ASTERISK-21857)
Review: https://reviewboard.asterisk.org/r/2621/



git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@393528 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This adds several unit tests for CEL functionality and provides the
requisite framework for creating additional unit tests.

This also cleans up some reference leaks that were occurring in
Stasis-Core message callback code.

Review: https://reviewboard.asterisk.org/r/2646/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@393410 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Originated channels are a bit odd - they are technically a dialed channel (thus
the party B or peer) but, since there is no caller, they are treated as the
party A. When entering into a bridge that already contains participants, the CDR
engine - if the CDR record is in the Dial state - attempts to match the person
entering the bridge with an existing participant. The idea is that if you dialed
someone and the person you dialed is already in the bridge, you don't need a new
CDR record, the existing CDR record describes the relationship.

Unfortunately, for an originated channel, there is no Party B. If no one was in
the bridge this didn't cause any issues; however, if participants were in the
bridge the CDR engine would attempt to match a non-existant Party B on the
channel's CDR record and explode.

This patch fixes that, and a unit test has been added to cover this case.



git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@393164 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@392779 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@392647 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Sorcery specific object information is now opaque and allocated with the object.
This means that modules do not need to be recompiled if the sorcery specific part
is changed. It also means that sorcery can store additional information on objects
and ensure it is freed or the reference count decreased when the object goes away.

To facilitate the above a generic sorcery allocator function has been added which
also ensures that allocated objects do not have a lock.

Extended fields have been added thanks to all of the above which allows specific fields
to be marked as extended, and thus simply stored as-is within the object. Type safety
is *NOT* enforced on these fields. A consumer of them has to query and ultimately perform
their own safety check. What does this mean? Extra modules can extend already defined
structures without having to modify them.

Tests have also been included to verify extended field functionality.

Review: https://reviewboard.asterisk.org/r/2585/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@392586 65c4cc65-6c06-0410-ace0-fbb531ad65f3