Commit 518bd1c0 authored by Dominik Charousset's avatar Dominik Charousset

Proofread new testing section for the manual

parent 0d30b74b
\section{Testing} \section{Testing}
\label{testing} \label{testing}
CAF has built-in support for writing unit tests. The approach of testing is declarative CAF comes with built-in support for writing unit tests in a domain-specific
and is very similar to the one in Boost.Test and Catch frameworks. language (DSL). The API looks similar to well-known testing frameworks such as
Boost.Test and Catch but adds CAF-specific macros for testing messaging between
actors.
There are four main concepts represented as macros in the testing framework. Our design leverages four main concepts:
\begin{itemize} \begin{itemize}
\item check - represents a single verification of boolean operation \item \textbf{Checks} represent single boolean expressions.
\item test - contains one or more checks \item \textbf{Tests} contain one or more checks.
\item suite - groups tests together \item \textbf{Fixtures} equip tests with a fixed data environment.
\item fixture - equips a test with fixed data environment \item \textbf{Suites} group tests together.
\end{itemize} \end{itemize}
Here is a very basic test case that captures the four main concepts described above. The following code illustrates a very basic test case that captures the four
main concepts described above.
\begin{lstlisting} \begin{lstlisting}
// Adds all tests in this compilation unit to the suite "math".
#define CAF_SUITE math
namespace n1 { // Pulls in all the necessary macros.
#include "caf/test/dsl.hpp"
namespace {
struct fixture {}; struct fixture {};
} // namespace
// Makes all members of `fixture` available to tests in the scope.
CAF_TEST_FIXTURE_SCOPE(math_tests, fixture) CAF_TEST_FIXTURE_SCOPE(math_tests, fixture)
// Implements our first test.
CAF_TEST(divide) { CAF_TEST(divide) {
CAF_CHECK(1 / 1 == 0); // this would fail CAF_CHECK(1 / 1 == 0); // fails
CAF_CHECK(2 / 2 == 1); // this would pass CAF_CHECK(2 / 2 == 1); // passes
CAF_REQUIRE(3 + 3 == 5); // this would fail and stop test execution [uncomment to try] CAF_REQUIRE(3 + 3 == 5); // fails and aborts test execution
CAF_CHECK(4 - 4 == 0); // You would not reach here because of failed REQUIRE CAF_CHECK(4 - 4 == 0); // unreachable due to previous requirement error
} }
CAF_TEST_FIXTURE_SCOPE_END() CAF_TEST_FIXTURE_SCOPE_END()
}
\end{lstlisting} \end{lstlisting}
We have used few macros such as \lstinline^CAF_CHECK^ and \lstinline^CAF_REQUIRE^ to validate our assertions. The main difference between The code above highlights the two basic macros \lstinline^CAF_CHECK^ and
\lstinline^CAF_REQUIRE^ and \lstinline^CAF_CHECK^ is that even if \lstinline^CAF_CHECK^ fails the control flow will continue, however failure of assertion \lstinline^CAF_REQUIRE^. The former reports failed checks, but allows the test
by \lstinline^CAF_REQUIRE^ will stop the test exeuction. to continue on error. The latter stops test execution if the boolean expression
evaluates to false.
The third macro worth mentioning is \lstinline^CAF_FAIL^. It unconditionally
stops test execution with an error message. This is particularly useful for
stopping program execution after receiving unexpected messages, as we will see
later.
\subsection{Testing Actors} \subsection{Testing Actors}
A simple example of a test case is shown below. This example shows that you can create the actor system in your fixture, spawn actors and send messages to them. The following example illustrates how to add an actor system as well as a
In other words, below code is not very different from your regular program however here we are using the macros such as \lstinline^CAF_CHECK^ and have arranged scoped actor to fixtures. This allows spawning of and interacting with actors
them as test cases. in a similar way regular programs would. Except that we are using macros such
as \lstinline^CAF_CHECK^ and provide tests rather than implementing a
\lstinline^caf_main^.
\begin{lstlisting} \begin{lstlisting}
namespace {
namespace n2 { struct fixture {
#define ERROR_HANDLER [&](caf::error &err) { CAF_FAIL(sys.render(err)); }
struct actor_fixture {
caf::actor_system_config cfg; caf::actor_system_config cfg;
caf::actor_system sys; caf::actor_system sys;
caf::scoped_actor self; caf::scoped_actor self;
actor_fixture() fixture() : sys(cfg), self(sys) {
: sys(cfg), // nop
self(sys) {} }
};
~actor_fixture() {}
};
caf::behavior adder(caf::event_based_actor *self) { caf::behavior adder() {
return { return {
[=](int x, int y) -> int { [=](int x, int y) {
return x+y; return x + y;
} }
}; };
} }
CAF_TEST_FIXTURE_SCOPE(actor_tests, actor_fixture) } // namespace
CAF_TEST(simple_actor_test) { CAF_TEST_FIXTURE_SCOPE(actor_tests, fixture)
auto adder_actor = sys.spawn(adder);
self->request(adder_actor, caf::infinite, 3, 4).receive([=](int r){ CAF_TEST(simple actor test) {
// Our Actor-Under-Test.
auto aut = self->spawn(adder);
self->request(aut, caf::infinite, 3, 4).receive(
[=](int r) {
CAF_CHECK(r == 7); CAF_CHECK(r == 7);
}, ERROR_HANDLER); },
} [&](caf::error& err) {
// Must not happen, stop test.
CAF_TEST_FIXTURE_SCOPE_END() CAF_FAIL(sys.render(err));
});
} }
CAF_TEST_FIXTURE_SCOPE_END()
\end{lstlisting} \end{lstlisting}
While the above example works, very soon you would start to face following problems - The example above works, but suffers from several issues:
\begin{itemize} \begin{itemize}
\item Significant amount of boilerplate \item
\item Above is a simple example of one actor, if you are unit testing one actor it would work however in most cases you would have an actor under test sending messages to
other actors. Writing code to validate all messages exchanges would not be trivial.
\item When testing the primary goal is to check the interaction between the actors and not necessarily the scheduling on multiple threads and/or the asynchronous nature of it.
\end{itemize} Significant amount of boilerplate code.
Next section describes the usage of \lstinline^test_coordinator^ that helps eliminiate the boilerplate as well as provide more deterministic and synchornous way \item
of writing tests.
\subsection{Test Coordinator} Using a scoped actor as illustrated above can only test one actor at a
time. However, messages between other actors are invisible to us.
CAF provides an implementation of coordinator (called \lstinline^test_coordinator^) that you supply to the scheduler. This coordinator is specifically designed \item
for testing as it does not schedule your actors on multiple thread.
There is also a fixture class called \lstinline^test_coordinator_fixture^ that is provided to hide the details and reduce the boilerplate for setting up the scheduler CAF runs actors in a thread pool by default. The resulting nondeterminism
with \lstinline^test_corrdinator^. makes triggering reliable ordering of messages near impossible. Further,
forcing timeouts to test error handling code is even harder.
\begin{lstlisting} \end{itemize}
namespace n3 {
using an_atom =
caf::atom_constant<caf::atom("an_atom")>;
caf::behavior ping(caf::event_based_actor* self) {
return {
[=](an_atom) -> std::string {
return "pong";
}
};
}
caf::behavior pong(caf::event_based_actor* self) {
return {
[=](an_atom, bool pang) -> std::string {
return pang ? "pang" : "ping";
}
};
}
struct ping_pong_fixture : test_coordinator_fixture<> {
};
CAF_TEST_FIXTURE_SCOPE(ping_pong_tests, ping_pong_fixture)
CAF_TEST(ping_should_return_pong) { \subsection{Deterministic Testing}
auto ping_actor = sys.spawn(ping);
self->send(ping_actor, an_atom::value); CAF provides a scheduler implementation specifically tailored for writing unit
tests called \lstinline^test_coordinator^. It does not start any threads and
instead gives unit tests full control over message dispatching and timeout
management.
// check if we sent it correctly To reduce boilerplate code, CAF also provides a fixture template called
expect((an_atom), from(self).to(ping_actor).with(an_atom::value)); \lstinline^test_coordinator_fixture^ that comes with ready-to-use actor system
// check the response we will get back (\lstinline^sys^) and testing scheduler (\lstinline^sched^). The optional
expect((std::string), from(ping_actor).to(self).with("pong")); template parameter allows unit tests to plugin custom actor system
} configuration classes.
Using this fixture unlocks three additional macros:
CAF_TEST(pong_should_return_ping_or_pang) { \begin{itemize}
auto pong_actor = sys.spawn(pong);
// check if we pass true that it should return pang
self->send(pong_actor, an_atom::value, true);
// check if we sent it correctly \item
expect((an_atom, bool), from(self).to(pong_actor).with(an_atom::value, true));
// check the response we will get back
expect((std::string), from(pong_actor).to(self).with("pang"));
\lstinline^expect^ checks for a single message. The macro verifies the
content types of the message and invokes the necessary member functions on
the test coordinator. Optionally, the macro checks the receiver of the
message and its content. If the expected message does not exist, the test
aborts.
// check if we pass false that it should return ping \item
self->send(pong_actor, an_atom::value, false);
// check if we sent it correctly \lstinline^allow^ is similar to \lstinline^expect^, but it does not abort
expect((an_atom, bool), from(self).to(pong_actor).with(an_atom::value, false)); the test if the expected message is missing. This macro returns
// check the response we will get back \lstinline^true^ if the allowed message was delivered, \lstinline^false^
expect((std::string), from(pong_actor).to(self).with("ping")); otherwise.
} \item
CAF_TEST_FIXTURE_SCOPE_END() \lstinline^disallow^ aborts the test if a particular message was delivered
to an actor.
} \end{itemize}
\end{lstlisting} The following example implements two actors, \lstinline^ping^ and
\lstinline^pong^, that exchange a configurable amount of messages. The test
\emph{three pings} then checks the contents of each message with
\lstinline^expect^ and verifies that no additional messages exist using
\lstinline^disallow^.
The above listings shows the declarative way testing of actors. \cppexample[12-65]{testing/ping_pong}
A call to \emph{expect} macro essentially schedules the run using the \lstinline^test_coordinator^.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment