........

Merged revisions 404764 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@404783 65c4cc65-6c06-0410-ace0-fbb531ad65f3

* Fixed several places where ao2_iterator_destroy() was not called.

* Fixed several iterator loop object variable reference problems.

* Fixed res_parking AMI actions returning non-zero.  Only the AMI logoff
action can return non-zero.

Review: https://reviewboard.asterisk.org/r/3087/
........

Merged revisions 404434 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@404436 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This fixes the unit tests that were broken by r403069 and several
functions requiring a new parameter for sanitization of JSON messages
generated from object snapshots.
........

Merged revisions 403094 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@403095 65c4cc65-6c06-0410-ace0-fbb531ad65f3

........
  r399887 | dlee | 2013-09-26 10:41:47 -0500 (Thu, 26 Sep 2013) | 1 line
  
  Minor performance bump by not allocate manager variable struct if we don't need it
........
  r400138 | dlee | 2013-09-30 10:24:00 -0500 (Mon, 30 Sep 2013) | 23 lines
  
  Stasis performance improvements
  
  This patch addresses several performance problems that were found in
  the initial performance testing of Asterisk 12.
  
  The Stasis dispatch object was allocated as an AO2 object, even though
  it has a very confined lifecycle. This was replaced with a straight
  ast_malloc().
  
  The Stasis message router was spending an inordinate amount of time
  searching hash tables. In this case, most of our routers had 6 or
  fewer routes in them to begin with. This was replaced with an array
  that's searched linearly for the route.
  
  We more heavily rely on AO2 objects in Asterisk 12, and the memset()
  in ao2_ref() actually became noticeable on the profile. This was
  #ifdef'ed to only run when AO2_DEBUG was enabled.
  
  After being misled by an erroneous comment in taskprocessor.c during
  profiling, the wrong comment was removed.
  
  Review: https://reviewboard.asterisk.org/r/2873/
........
  r400178 | dlee | 2013-09-30 13:26:27 -0500 (Mon, 30 Sep 2013) | 24 lines
  
  Taskprocessor optimization; switch Stasis to use taskprocessors
  
  This patch optimizes taskprocessor to use a semaphore for signaling,
  which the OS can do a better job at managing contention and waiting
  that we can with a mutex and condition.
  
  The taskprocessor execution was also slightly optimized to reduce the
  number of locks taken.
  
  The only observable difference in the taskprocessor implementation is
  that when the final reference to the taskprocessor goes away, it will
  execute all tasks to completion instead of discarding the unexecuted
  tasks.
  
  For systems where unnamed semaphores are not supported, a really
  simple semaphore implementation is provided. (Which gives identical
  performance as the original taskprocessor implementation).
  
  The way we ended up implementing Stasis caused the threadpool to be a
  burden instead of a boost to performance. This was switched to just
  use taskprocessors directly for subscriptions.
  
  Review: https://reviewboard.asterisk.org/r/2881/
........
  r400180 | dlee | 2013-09-30 13:39:34 -0500 (Mon, 30 Sep 2013) | 28 lines
  
  Optimize how Stasis forwards are dispatched
  
  This patch optimizes how forwards are dispatched in Stasis.
  
  Originally, forwards were dispatched as subscriptions that are invoked
  on the publishing thread. This did not account for the vast number of
  forwards we would end up having in the system, and the amount of work it
  would take to walk though the forward subscriptions.
  
  This patch modifies Stasis so that rather than walking the tree of
  forwards on every dispatch, when forwards and subscriptions are changed,
  the subscriber list for every topic in the tree is changed.
  
  This has a couple of benefits. First, this reduces the workload of
  dispatching messages. It also reduces contention when dispatching to
  different topics that happen to forward to the same aggregation topic
  (as happens with all of the channel, bridge and endpoint topics).
  
  Since forwards are no longer subscriptions, the bulk of this patch is
  simply changing stasis_subscription objects to stasis_forward objects
  (which, admittedly, I should have done in the first place.)
  
  Since this required me to yet again put in a growing array, I finally
  abstracted that out into a set of ast_vector macros in
  asterisk/vector.h.
  
  Review: https://reviewboard.asterisk.org/r/2883/
........
  r400181 | dlee | 2013-09-30 13:48:57 -0500 (Mon, 30 Sep 2013) | 28 lines
  
  Remove dispatch object allocation from Stasis publishing
  
  While looking for areas for performance improvement, I realized that an
  unused feature in Stasis was negatively impacting performance.
  
  When a message is sent to a subscriber, a dispatch object is allocated
  for the dispatch, containing the topic the message was published to, the
  subscriber the message is being sent to, and the message itself.
  
  The topic is actually unused by any subscriber in Asterisk today. And
  the subscriber is associated with the taskprocessor the message is being
  dispatched to.
  
  First, this patch removes the unused topic parameter from Stasis
  subscription callbacks.
  
  Second, this patch introduces the concept of taskprocessor local data,
  data that may be set on a taskprocessor and provided along with the data
  pointer when a task is pushed using the ast_taskprocessor_push_local()
  call. This allows the task to have both data specific to that
  taskprocessor, in addition to data specific to that invocation.
  
  With those two changes, the dispatch object can be removed completely,
  and the message is simply refcounted and sent directly to the
  taskprocessor.
  
  Review: https://reviewboard.asterisk.org/r/2884/
........

Merged revisions 399887,400138,400178,400180-400181 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@400186 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Change r395954 reordered some stasis object destruction, which should
have been fine. Unfortunately, it caused some hard to reproduce issues
related to objects being accessed after they had been destroyed. The
patch in r396329 fixed the destruction order problem; this patch
addresses the underlying issue. A few other stasis-related fixes were
also added.

 * Add ref-bumps around areas where objects may get transitively
   destroyed. (For example, where we lock a topic, unref a subscription,
   which unrefs the topic, which explodes the topic when we try to
   unlock it.)

 * Wrote an extensive doxygen page about Stasis implementation,
   relationships between objects, lifecycles of objects, how the
   refcounting works, etc. Many other comments were added, corrected, or
   cleaned up.

 * Added an assert to the topic dtor to catch extra ref decrements.

 * Fixed type used after destruction errors for graceful shutdown in
   stasis_channels.c.

 * I added two unit tests in an attempt to catch destruction order
   issues. Since the underlying cause is a race condition, though, the
   tests rarely failed even when the code was wrong.

 * Fixed a leak in stasis_cache_pattern.c.

(closes issue ASTERISK-22243)
Review: https://reviewboard.asterisk.org/r/2746/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396842 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396480 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396463 65c4cc65-6c06-0410-ace0-fbb531ad65f3

The Stasis changes in r395954 had an unanticipated side effect: messages
published directly to an _all topic does not get forwarded to the
corresponding caching topic.

This patch fixes that by changing how caching topics forward messages,
and how the caching pattern forwards are setup.

For the caching pattern, the all_topic is forwarded to the
all_topic_cached. This forwards messages published directly to the
all_topic to all_topic_cached.

In order to avoid duplicate messages on all_topic_cached, caching topics
were changed to no longer forward uncached messages. Subscribers to an
individual caching topic should only expect to receive cache updates,
and subscription change messages. Since individual caching topics are
new, this shouldn't be a problem.

There are a few minor changes to the pre-cache split behavior.

 * For topics changed to use the caching pattern, the all_topic_cached
   will forward snapshots in addition to cache updates. Since
   subscribers by design ignore unexpected messages, this should be
   fine.

 * Caching topics that don't use the caching pattern no longer forward
   non-cache updates. This makes no difference for the current caching
   topics.

   * mwi_topic_cached, channel_by_name_topic and
     presence_state_topic_cached have no subscribers

   * device_state_topic_cached's only subscriber only processes cache
     udpates

(issue ASTERISK-22243)
Review: https://reviewboard.asterisk.org/r/2738


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@396329 65c4cc65-6c06-0410-ace0-fbb531ad65f3

In working with res_stasis, I discovered a significant limitation to
the current structure of stasis_caching_topics: you cannot subscribe
to cache updates for a single channel/bridge/endpoint/etc.

To address this, this patch splits the cache away from the
stasis_caching_topic, making it a first class object. The stasis_cache
object is shared amongst individual stasis_caching_topics that are
created per channel/endpoint/etc. These are still forwarded to global
whatever_all_cached topics, so their use from most of the code does
not change.

In making these changes, I noticed that we frequently used a similar
pattern for bridges, endpoints and channels:

     single_topic  ---------------->  all_topic
           ^
           |
     single_topic_cached  ----+---->  all_topic_cached
                              |
                              +---->  cache

This pattern was extracted as the 'Stasis Caching Pattern', defined in
stasis_caching_pattern.h. This avoids a lot of duplicate code between
the different domain objects.

Since the cache is now disassociated from its upstream caching topics,
this also necessitated a change to how the 'guaranteed' flag worked
for retrieving from a cache. The code for handling the caching
guarantee was extracted into a 'stasis_topic_wait' function, which
works for any stasis_topic.

(closes issue ASTERISK-22002)
Review: https://reviewboard.asterisk.org/r/2672/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395954 65c4cc65-6c06-0410-ace0-fbb531ad65f3

The stasis_cache_update messages are somewhat cumbersome to handle
with the stasis_message_router. Since all updates have the same
message type, they are normally handled with the same route.

Since caching itself is a first class component of stasis-core, it
makes sense for the router to handle the cache update messages itself.
This patch adds stasis_message_router_add_cache_update() and
stasis_message_router_remove_cache_update() to handle the routing of
stasis_cache_update messages.

This patch also corrects an issue with manager_{bridging,channels}.c,
where events might be reordered. The reordering occurs because the
components use different message routers, which they needed because
they both needed to route cache update messages. They now both use
manager's router, and add cache routes for just the cache updates they
are interested in.

(closes issue ASTERISK-22038)
Review: https://reviewboard.asterisk.org/r/2677/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395118 65c4cc65-6c06-0410-ace0-fbb531ad65f3

When a Stasis message type is defined in a loadable module, handling
those messages for AMI and res_stasis events can be cumbersome.

This patch adds a vtable to stasis_message_type, with to_ami and
to_json virtual functions. These allow messages to be handled
abstractly without putting module-specific code in core.

As an example, the VarSet AMI event was refactored to use the to_ami
virtual function.

(closes issue ASTERISK-21817)
Review: https://reviewboard.asterisk.org/r/2579/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@391403 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Stasis cache clear message payloads now consist of a stasis_message
representative of the message to be cleared from the cache. This allows
multiple parallel caches to coexist and be cleared properly by the same
cache clear message even when keyed on different fields.

This change fixes a bug where multiple cache clears could be posted for
channels. The cache clear is now produced in the destructor instead of
ast_hangup.

Additionally, dummy channels are no longer capable of producing channel
snapshots.

Review: https://reviewboard.asterisk.org/r/2596


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@390830 65c4cc65-6c06-0410-ace0-fbb531ad65f3

The API itself is documented using Swagger, a lightweight mechanism for
documenting RESTful API's using JSON. This allows us to use swagger-ui
to provide executable documentation for the API, generate client
bindings in different languages, and generate a lot of the boilerplate
code for implementing the RESTful bindings. The API docs live in the
rest-api/ directory.

The RESTful bindings are generated from the Swagger API docs using a set
of Mustache templates.  The code generator is written in Python, and
uses Pystache. Pystache has no dependencies, and be installed easily
using pip. Code generation code lives in rest-api-templates/.

The generated code reduces a lot of boilerplate when it comes to
handling HTTP requests. It also helps us have greater consistency in the
REST API.

(closes issue ASTERISK-20891)
Review: https://reviewboard.asterisk.org/r/2376/

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@386232 65c4cc65-6c06-0410-ace0-fbb531ad65f3

This patch fixes an issue of message ordering that occurs when
multiple topics are forwarded to an aggregator topic (such as
ast_channel_topic_all()).

It is (very reasonably) expected that the rules governing message
dispatch order still apply, so long as the messages start from the
same thread, and are received by the same subscription. Because the
existing code had an additional layer of dispatching via the Stasis
thread pool for forwards, those promises couldn't be kept.

Forwarding subscriptions no longer have their own mailbox, and now
dispatch directly from the forwarding topic's stasis_publish()
call. This means that the topic's lock is held for the duration of not
only a message's dispatch, but the dispatch of all the forwards. This
shouldn't be a problem right now, but if an aggregator topic had many
subscribers, it could become a problem. But I figure we can write more
clever code when the time comes, if necessary.

Review: https://reviewboard.asterisk.org/r/2419/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@384413 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@384261 65c4cc65-6c06-0410-ace0-fbb531ad65f3

Often times, when subscribing to a topic, one wants to handle
different message types differently. While one could cascade if/else
statements through the subscription handler, it is much cleaner to
specify a different callback for each message type. The
stasis_message_router is here to help!

A stasis_message_router is constructed for a particular stasis_topic,
which is subscribes to. Call stasis_message_router_unsubscribe() to
cancel that subscription.

Once constructed, routes can be added using
stasis_message_router_add() (or stasis_message_router_set_default()
for any messages not handled by other routes). There may be only one
route per stasis_message_type. The route's callback is invoked just as
if it were a callback for a subscription; but it only gets called for
messages of the specified type.

(issue ASTERISK-20887)
Review: https://reviewboard.asterisk.org/r/2390/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@383242 65c4cc65-6c06-0410-ace0-fbb531ad65f3

git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@383169 65c4cc65-6c06-0410-ace0-fbb531ad65f3

The cache dump mechanism allows the developer to retreive multiple
items of a given type (or of all types) from the cache residing in a
stasis caching topic in addition to the existing single-item cache
retreival mechanism.  This also adds to the caching unit tests to
ensure that the new cache dump mechanism is functioning properly.

Review: https://reviewboard.asterisk.org/r/2367/
(issue ASTERISK-21097)


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@382705 65c4cc65-6c06-0410-ace0-fbb531ad65f3

For the initial use of this bus, I took some work kmoore did creating
channel snapshots. So rather than create AMI events directly in the
channel code, this patch generates Stasis events, which manager.c uses
to then publish the AMI event.

This message bus provides a generic publish/subscribe mechanism within
Asterisk. This message bus is:

 - Loosely coupled; new message types can be added in seperate modules.
 - Easy to use; publishing and subscribing are straightforward
   operations.

In addition to basic publish/subscribe, the patch also provides
mechanisms for message forwarding, and for message caching.

(issue ASTERISK-20887)
(closes issue ASTERISK-20959)
Review: https://reviewboard.asterisk.org/r/2339/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@382685 65c4cc65-6c06-0410-ace0-fbb531ad65f3