multi_pass.qbk 33 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723
  1. [/==============================================================================
  2. Copyright (C) 2001-2011 Hartmut Kaiser
  3. Copyright (C) 2001-2011 Joel de Guzman
  4. Copyright (C) 2001-2002 Daniel C. Nuffer
  5. Distributed under the Boost Software License, Version 1.0. (See accompanying
  6. file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
  7. ===============================================================================/]
  8. [section:multi_pass The multi pass iterator]
  9. Backtracking in __qi__ requires the use of the following types of iterator:
  10. forward, bidirectional, or random access. Because of backtracking, input
  11. iterators cannot be used. Therefore, the standard library classes
  12. `std::istreambuf_iterator` and `std::istream_iterator`, that fall under the
  13. category of input iterators, cannot be used. Another input iterator that is of
  14. interest is one that wraps a lexer, such as LEX.
  15. [note In general, __qi__ generates recursive descent parser which require
  16. backtracking parsers by design. For this reason we need to provide at
  17. least forward iterators to any of __qi__'s API functions. This is not an
  18. absolute requirement though. In the future, we shall see more
  19. deterministic parsers that require no more than 1 character (token) of
  20. lookahead. Such parsers allow us to use input iterators such as the
  21. `std::istream_iterator` as is. ]
  22. Backtracking can be implemented only if we are allowed to save an iterator
  23. position, i.e. making a copy of the current iterator. Unfortunately, with an
  24. input iterator, there is no way to do so, and thus input iterators will not
  25. work with backtracking in __qi__. One solution to this problem is to simply
  26. load all the data to be parsed into a container, such as a vector or deque,
  27. and then pass the begin and end of the container to __qi__. This method can be
  28. too memory intensive for certain applications, which is why the `multi_pass`
  29. iterator was created.
  30. [heading Using the multi_pass]
  31. The `multi_pass` iterator will convert any input iterator into a forward
  32. iterator suitable for use with __qi__. `multi_pass` will buffer data when
  33. needed and will discard the buffer when its contents is not needed anymore.
  34. This happens either if only one copy of the iterator exists or if no
  35. backtracking can occur.
  36. A grammar must be designed with care if the `multi_pass` iterator is used.
  37. Any rule that may need to backtrack, such as one that contains an alternative,
  38. will cause data to be buffered. The rules that are optimal to use are
  39. repetition constructs (as kleene and plus).
  40. Sequences of the form `a >> b` will buffer data as well. This is different from
  41. the behavior of __classic__ but for a good reason. Sequences need to reset the
  42. current iterator to its initial state if one of the components of a sequence
  43. fails to match. To compensate for this behavior we added functionality to
  44. the `expect` parsers (i.e. constructs like `a > b`). Expectation points introduce
  45. deterministic points into the grammar ensuring no backtracking can occur if
  46. they match. For this reason we clear the buffers of any multi_pass iterator
  47. on each expectation point, ensuring minimal buffer content even for large
  48. grammars.
  49. [important If you use an error handler in conjunction with the `expect` parser
  50. while utilizing a `multi_pass` iterator and you intend to use
  51. the error handler to force a `retry` or a `fail` (see the
  52. description of error handlers - __fixme__: insert link), then
  53. you need to instantiate the error handler using `retry` or `fail`,
  54. for instance:
  55. ``
  56. rule r<iterator_type> r;
  57. on_error<retry>(r, std::cout << phoenix::val("Error!"));
  58. ``
  59. If you fail to do so the resulting code will trigger an assert
  60. statement at runtime.]
  61. Any rule that repeats, such as kleene_star (`*a`) or positive such as (`+a`),
  62. will only buffer the data for the current repetition.
  63. In typical grammars, ambiguity and therefore lookahead is often localized. In
  64. fact, many well designed languages are fully deterministic and require no
  65. lookahead at all. Peeking at the first character from the input will
  66. immediately determine the alternative branch to take. Yet, even with highly
  67. ambiguous grammars, alternatives are often of the form `*(a | b | c | d)`.
  68. The input iterator moves on and is never stuck at the beginning. Let's look at
  69. a Pascal snippet for example:
  70. program =
  71. programHeading >> block >> '.'
  72. ;
  73. block =
  74. *( labelDeclarationPart
  75. | constantDefinitionPart
  76. | typeDefinitionPart
  77. | variableDeclarationPart
  78. | procedureAndFunctionDeclarationPart
  79. )
  80. >> statementPart
  81. ;
  82. Notice the alternatives inside the Kleene star in the rule block . The rule
  83. gobbles the input in a linear manner and throws away the past history with each
  84. iteration. As this is fully deterministic LL(1) grammar, each failed
  85. alternative only has to peek 1 character (token). The alternative that consumes
  86. more than 1 character (token) is definitely a winner. After which, the Kleene
  87. star moves on to the next.
  88. Now, after the lecture on the features to be careful with when using
  89. `multi_pass`, you may think that `multi_pass` is way too restrictive to use.
  90. That's not the case. If your grammar is deterministic, you can make use of
  91. the `flush_multi_pass` pseudo parser in your grammar to ensure that data is not
  92. buffered when unnecessary (`flush_multi_pass` is available from the __qi__
  93. parser __repo__).
  94. Here we present a minimal example showing a minimal use case. The `multi_pass`
  95. iterator is highly configurable, but the default policies have been chosen so
  96. that its easily usable with input iterators such as `std::istreambuf_iterator`.
  97. For the complete source code of this example please refer to
  98. [@../../example/support/multi_pass.cpp multi_pass.cpp].
  99. [import ../../example/support/multi_pass.cpp]
  100. [tutorial_multi_pass]
  101. [heading Using the flush_multi_pass parser]
  102. The __spirit__ __repo__ contains the `flush_multi_pass` parser component.
  103. This is usable in conjunction with the `multi_pass` iterator to minimize the
  104. buffering. It allows to insert explicit synchronization points into your
  105. grammar where it is safe to clear any stored input as it is ensured that no
  106. backtracking can occur at this point anymore.
  107. When the `flush_multi_pass` parser is used with `multi_pass`, it will call
  108. `multi_pass::clear_queue()`. This will cause any buffered data to be erased.
  109. This also will invalidate all other copies of multi_pass and they should not
  110. be used. If they are, an `boost::illegal_backtracking` exception will be
  111. thrown.
  112. [heading The multi_pass Policies]
  113. The `multi_pass` iterator is a templated class configurable using policies.
  114. The description of `multi_pass` above is how it was originally implemented
  115. (before it used policies), and is the default configuration now. But,
  116. `multi_pass` is capable of much more. Because of the open-ended nature of
  117. policies, you can write your own policy to make `multi_pass` behave in a way
  118. that we never before imagined.
  119. The multi_pass class has two template parameters:
  120. [variablelist The multi_pass template parameters
  121. [[Input] [The type multi_pass uses to acquire it's input. This is
  122. typically an input iterator, or functor.]]
  123. [[Policies] [The combined policies to use to create an instance of a
  124. multi_pass iterator. This combined policy type is described
  125. below]]
  126. ]
  127. It is possible to implement all of the required functionality of the combined
  128. policy in a single class. But it has shown to be more convenient to split this
  129. into four different groups of functions, i.e. four separate, but well
  130. coordinated policies. For this reason the `multi_pass` library
  131. implements a template `iterator_policies::default_policy` allowing to combine
  132. several different policies, each implementing one of the functionality groups:
  133. [table Policies needed for default_policy template
  134. [[Template Parameter] [Description]]
  135. [[`OwnershipPolicy`] [This policy determines how `multi_pass` deals with
  136. it's shared components.]]
  137. [[`CheckingPolicy`] [This policy determines how checking for invalid
  138. iterators is done.]]
  139. [[`InputPolicy`] [A class that defines how `multi_pass` acquires its
  140. input. The `InputPolicy` is parameterized by the
  141. `Input` template parameter to the `multi_pass`.]]
  142. [[`StoragePolicy`] [The buffering scheme used by `multi_pass` is
  143. determined and managed by the StoragePolicy.]]
  144. ]
  145. The `multi_pass` library contains several predefined policy implementations
  146. for each of the policy types as described above. First we will describe those
  147. predefined types. Afterwards we will give some guidelines how you can write
  148. your own policy implementations.
  149. [heading Predefined policies]
  150. All predefined `multi_pass` policies are defined in the namespace
  151. `boost::spirit::iterator_policies`.
  152. [table Predefined policy classes
  153. [[Class name] [Description]]
  154. [[*InputPolicy* classes]]
  155. [[`input_iterator`] [This policy directs `multi_pass` to read from an
  156. input iterator of type `Input`.]]
  157. [[`buffering_input_iterator`] [This policy directs `multi_pass` to read from an
  158. input iterator of type `Input`. Additionally it buffers
  159. the last character received from the underlying iterator.
  160. This allows to wrap iterators not buffering the last
  161. character on their own (as `std::istreambuf_iterator`).]]
  162. [[`istream`] [This policy directs `multi_pass` to read from an
  163. input stream of type `Input` (usually a
  164. `std::basic_istream`).]]
  165. [[`lex_input`] [This policy obtains it's input by calling yylex(),
  166. which would typically be provided by a scanner
  167. generated by __flex__. If you use this policy your code
  168. must link against a __flex__ generated scanner.]]
  169. [[`functor_input`] [This input policy obtains it's data by calling a
  170. functor of type `Input`. The functor must meet
  171. certain requirements. It must have a typedef called
  172. `result_type` which should be the type returned
  173. from `operator()`. Also, since an input policy needs
  174. a way to determine when the end of input has been
  175. reached, the functor must contain a static variable
  176. named `eof` which is comparable to a variable of
  177. `result_type`.]]
  178. [[`split_functor_input`][This is essentially the same as the `functor_input`
  179. policy except that the (user supplied) function
  180. object exposes separate `unique` and `shared` sub
  181. classes, allowing to integrate the functors /unique/
  182. data members with the `multi_pass` data items held
  183. by each instance and its /shared/ data members will
  184. be integrated with the `multi_pass` members shared
  185. by all copies.]]
  186. [[*OwnershipPolicy* classes]]
  187. [[`ref_counted`] [This class uses a reference counting scheme.
  188. The `multi_pass` will delete it's shared components
  189. when the count reaches zero.]]
  190. [[`first_owner`] [When this policy is used, the first `multi_pass`
  191. created will be the one that deletes the shared data.
  192. Each copy will not take ownership of the shared data.
  193. This works well for __spirit__, since no dynamic
  194. allocation of iterators is done. All copies are made
  195. on the stack, so the original iterator has the
  196. longest lifespan.]]
  197. [[*CheckingPolicy* classes]]
  198. [[`no_check`] [This policy does no checking at all.]]
  199. [[`buf_id_check`] [This policy keeps around a buffer id, or a buffer
  200. age. Every time `clear_queue()` is called on a
  201. `multi_pass` iterator, it is possible that all other
  202. iterators become invalid. When `clear_queue()` is
  203. called, `buf_id_check` increments the buffer id.
  204. When an iterator is dereferenced, this policy checks
  205. that the buffer id of the iterator matches the shared
  206. buffer id. This policy is most effective when used
  207. together with the `split_std_deque` StoragePolicy.
  208. It should not be used with the `fixed_size_queue`
  209. StoragePolicy, because it will not detect iterator
  210. dereferences that are out of range.]]
  211. [[full_check] [This policy has not been implemented yet. When it
  212. is, it will keep track of all iterators and make
  213. sure that they are all valid. This will be mostly
  214. useful for debugging purposes as it will incur
  215. significant overhead.]]
  216. [[*StoragePolicy* classes]]
  217. [[`split_std_deque`] [Despite its name this policy keeps all buffered data
  218. in a `std::vector`. All data is stored as long as
  219. there is more than one iterator. Once the iterator
  220. count goes down to one, and the queue is no longer
  221. needed, it is cleared, freeing up memory. The queue
  222. can also be forcibly cleared by calling
  223. `multi_pass::clear_queue()`.]]
  224. [[`fixed_size_queue<N>`][This policy keeps a circular buffer that is size
  225. `N+1` and stores `N` elements. `fixed_size_queue`
  226. is a template with a `std::size_t` parameter that
  227. specified the queue size. It is your responsibility
  228. to ensure that `N` is big enough for your parser.
  229. Whenever the foremost iterator is incremented, the
  230. last character of the buffer is automatically
  231. erased. Currently there is no way to tell if an
  232. iterator is trailing too far behind and has become
  233. invalid. No dynamic allocation is done by this
  234. policy during normal iterator operation, only on
  235. initial construction. The memory usage of this
  236. `StoragePolicy` is set at `N+1` bytes, unlike
  237. `split_std_deque`, which is unbounded.]]
  238. ]
  239. [heading Combinations: How to specify your own custom multi_pass]
  240. The beauty of policy based designs is that you can mix and match policies to
  241. create your own custom iterator by selecting the policies you want. Here's an
  242. example of how to specify a custom `multi_pass` that wraps an
  243. `std::istream_iterator<char>`, and is slightly more efficient than the default
  244. `multi_pass` (as generated by the `make_default_multi_pass()` API function)
  245. because it uses the `iterator_policies::first_owner` OwnershipPolicy and the
  246. `iterator_policies::no_check` CheckingPolicy:
  247. typedef multi_pass<
  248. std::istream_iterator<char>
  249. , iterator_policies::default_policy<
  250. iterator_policies::first_owner
  251. , iterator_policies::no_check
  252. , iterator_policies::buffering_input_iterator
  253. , iterator_policies::split_std_deque
  254. >
  255. > first_owner_multi_pass_type;
  256. The default template parameters for `iterator_policies::default_policy` are:
  257. * `iterator_policies::ref_counted` OwnershipPolicy
  258. * `iterator_policies::no_check` CheckingPolicy, if `BOOST_SPIRIT_DEBUG` is
  259. defined: `iterator_policies::buf_id_check` CheckingPolicy
  260. * `iterator_policies::buffering_input_iterator` InputPolicy, and
  261. * `iterator_policies::split_std_deque` StoragePolicy.
  262. So if you use `multi_pass<std::istream_iterator<char> >` you will get those
  263. pre-defined behaviors while wrapping an `std::istream_iterator<char>`.
  264. [heading Dealing with constant look ahead]
  265. There is one other pre-defined class called `look_ahead`. The class
  266. `look_ahead` is another predefine `multi_pass` iterator type. It has two
  267. template parameters: `Input`, the type of the input iterator to wrap, and a
  268. `std::size_t N`, which specifies the size of the buffer to the
  269. `fixed_size_queue` policy. While the default multi_pass configuration is
  270. designed for safety, `look_ahead` is designed for speed. `look_ahead` is derived
  271. from a multi_pass with the following policies: `input_iterator` InputPolicy,
  272. `first_owner` OwnershipPolicy, `no_check` CheckingPolicy, and
  273. `fixed_size_queue<N>` StoragePolicy.
  274. This iterator is defined by including the files:
  275. // forwards to <boost/spirit/home/support/look_ahead.hpp>
  276. #include <boost/spirit/include/support_look_ahead.hpp>
  277. Also, see __include_structure__.
  278. [heading Reading from standard input streams]
  279. Yet another predefined iterator for wrapping standard input streams (usually a
  280. `std::basic_istream<>`) is called `basic_istream_iterator<Char, Traits>`. This
  281. class is usable as a drop in replacement for `std::istream_iterator<Char, Traits>`.
  282. Its only difference is that it is a forward iterator (instead of the
  283. `std::istream_iterator`, which is an input iterator). `basic_istream_iterator`
  284. is derived from a multi_pass with the following policies: `istream` InputPolicy,
  285. `ref_counted` OwnershipPolicy, `no_check` CheckingPolicy, and
  286. `split_std_deque` StoragePolicy.
  287. There exists an additional predefined typedef:
  288. typedef basic_istream_iterator<char, std::char_traits<char> > istream_iterator;
  289. This iterator is defined by including the files:
  290. // forwards to <boost/spirit/home/support/istream_iterator.hpp>
  291. #include <boost/spirit/include/support_istream_iterator.hpp>
  292. Also, see __include_structure__.
  293. [heading How to write a functor for use with the `functor_input` InputPolicy]
  294. If you want to use the `functor_input` InputPolicy, you can write your own
  295. function object that will supply the input to `multi_pass`. The function object
  296. must satisfy several requirements. It must have a typedef `result_type` which
  297. specifies the return type of its `operator()`. This is standard practice in the
  298. STL. Also, it must supply a static variable called eof which is compared against
  299. to know whether the input has reached the end. Last but not least the function
  300. object must be default constructible. Here is an example:
  301. #include <iostream>
  302. #include <boost/spirit/home/qi.hpp>
  303. #include <boost/spirit/home/support.hpp>
  304. #include <boost/spirit/home/support/multi_pass.hpp>
  305. #include <boost/spirit/home/support/iterators/detail/functor_input_policy.hpp>
  306. // define the function object
  307. class iterate_a2m
  308. {
  309. public:
  310. typedef char result_type;
  311. iterate_a2m() : c_('A') {}
  312. iterate_a2m(char c) : c_(c) {}
  313. result_type operator()()
  314. {
  315. if (c_ == 'M')
  316. return eof;
  317. return c_++;
  318. }
  319. static result_type eof;
  320. private:
  321. char c_;
  322. };
  323. iterate_a2m::result_type iterate_a2m::eof = iterate_a2m::result_type('M');
  324. using namespace boost::spirit;
  325. // create two iterators using the define function object, one of which is
  326. // an end iterator
  327. typedef multi_pass<iterate_a2m
  328. , iterator_policies::first_owner
  329. , iterator_policies::no_check
  330. , iterator_policies::functor_input
  331. , iterator_policies::split_std_deque>
  332. functor_multi_pass_type;
  333. int main()
  334. {
  335. functor_multi_pass_type first = functor_multi_pass_type(iterate_a2m());
  336. functor_multi_pass_type last;
  337. // use the iterators: this will print "ABCDEFGHIJKL"
  338. while (first != last) {
  339. std::cout << *first;
  340. ++first;
  341. }
  342. std::cout << std::endl;
  343. return 0;
  344. }
  345. [heading How to write policies for use with multi_pass]
  346. All policies to be used with the `default_policy` template need to have two
  347. embedded classes: `unique` and `shared`. The `unique` class needs to implement
  348. all required functions for a particular policy type. In addition it may hold
  349. all member data items being /unique/ for a particular instance of a `multi_pass`
  350. (hence the name). The `shared` class does not expose any member functions
  351. (except sometimes a constructor), but it may hold all member data items to be
  352. /shared/ between all copies of a particular `multi_pass`.
  353. [heading InputPolicy]
  354. An `InputPolicy` must have the following interface:
  355. struct input_policy
  356. {
  357. // Input is the same type used as the first template parameter
  358. // while instantiating the multi_pass
  359. template <typename Input>
  360. struct unique
  361. {
  362. // these typedef's will be exposed as the multi_pass iterator
  363. // properties
  364. typedef __unspecified_type__ value_type;
  365. typedef __unspecified_type__ difference_type;
  366. typedef __unspecified_type__ distance_type;
  367. typedef __unspecified_type__ pointer;
  368. typedef __unspecified_type__ reference;
  369. unique() {}
  370. explicit unique(Input) {}
  371. // destroy is called whenever the last copy of a multi_pass is
  372. // destructed (ownership_policy::release() returned true)
  373. //
  374. // mp: is a reference to the whole multi_pass instance
  375. template <typename MultiPass>
  376. static void destroy(MultiPass& mp);
  377. // swap is called by multi_pass::swap()
  378. void swap(unique&);
  379. // get_input is called whenever the next input character/token
  380. // should be fetched.
  381. //
  382. // mp: is a reference to the whole multi_pass instance
  383. //
  384. // This method is expected to return a reference to the next
  385. // character/token
  386. template <typename MultiPass>
  387. static typename MultiPass::reference get_input(MultiPass& mp);
  388. // advance_input is called whenever the underlying input stream
  389. // should be advanced so that the next call to get_input will be
  390. // able to return the next input character/token
  391. //
  392. // mp: is a reference to the whole multi_pass instance
  393. template <typename MultiPass>
  394. static void advance_input(MultiPass& mp);
  395. // input_at_eof is called to test whether this instance is a
  396. // end of input iterator.
  397. //
  398. // mp: is a reference to the whole multi_pass instance
  399. //
  400. // This method is expected to return true if the end of input is
  401. // reached. It is often used in the implementation of the function
  402. // storage_policy::is_eof.
  403. template <typename MultiPass>
  404. static bool input_at_eof(MultiPass const& mp);
  405. // input_is_valid is called to verify if the parameter t represents
  406. // a valid input character/token
  407. //
  408. // mp: is a reference to the whole multi_pass instance
  409. // t: is the character/token to test for validity
  410. //
  411. // This method is expected to return true if the parameter t
  412. // represents a valid character/token.
  413. template <typename MultiPass>
  414. static bool input_is_valid(MultiPass const& mp, value_type const& t);
  415. };
  416. // Input is the same type used as the first template parameter passed
  417. // while instantiating the multi_pass
  418. template <typename Input>
  419. struct shared
  420. {
  421. explicit shared(Input) {}
  422. };
  423. };
  424. It is possible to derive the struct `unique` from the type
  425. `boost::spirit::detail::default_input_policy`. This type implements a minimal
  426. sufficient interface for some of the required functions, simplifying the task
  427. of writing a new input policy.
  428. This class may implement a function `destroy()` being called during destruction
  429. of the last copy of a `multi_pass`. This function should be used to free any of
  430. the shared data items the policy might have allocated during construction of
  431. its `shared` part. Because of the way `multi_pass` is implemented any allocated
  432. data members in `shared` should _not_ be deep copied in a copy constructor of
  433. `shared`.
  434. [heading OwnershipPolicy]
  435. The `OwnershipPolicy` must have the following interface:
  436. struct ownership_policy
  437. {
  438. struct unique
  439. {
  440. // destroy is called whenever the last copy of a multi_pass is
  441. // destructed (ownership_policy::release() returned true)
  442. //
  443. // mp: is a reference to the whole multi_pass instance
  444. template <typename MultiPass>
  445. static void destroy(MultiPass& mp);
  446. // swap is called by multi_pass::swap()
  447. void swap(unique&);
  448. // clone is called whenever a multi_pass is copied
  449. //
  450. // mp: is a reference to the whole multi_pass instance
  451. template <typename MultiPass>
  452. static void clone(MultiPass& mp);
  453. // release is called whenever a multi_pass is destroyed
  454. //
  455. // mp: is a reference to the whole multi_pass instance
  456. //
  457. // The method is expected to return true if the destructed
  458. // instance is the last copy of a particular multi_pass.
  459. template <typename MultiPass>
  460. static bool release(MultiPass& mp);
  461. // is_unique is called to test whether this instance is the only
  462. // existing copy of a particular multi_pass
  463. //
  464. // mp: is a reference to the whole multi_pass instance
  465. //
  466. // The method is expected to return true if this instance is unique
  467. // (no other copies of this multi_pass exist).
  468. template <typename MultiPass>
  469. static bool is_unique(MultiPass const& mp);
  470. };
  471. struct shared {};
  472. };
  473. It is possible to derive the struct `unique` from the type
  474. `boost::spirit::detail::default_ownership_policy`. This type implements a
  475. minimal sufficient interface for some of the required functions, simplifying
  476. the task of writing a new ownership policy.
  477. This class may implement a function `destroy()` being called during destruction
  478. of the last copy of a `multi_pass`. This function should be used to free any of
  479. the shared data items the policy might have allocated during construction of
  480. its `shared` part. Because of the way `multi_pass` is implemented any allocated
  481. data members in `shared` should _not_ be deep copied in a copy constructor of
  482. `shared`.
  483. [heading CheckingPolicy]
  484. The `CheckingPolicy` must have the following interface:
  485. struct checking_policy
  486. {
  487. struct unique
  488. {
  489. // swap is called by multi_pass::swap()
  490. void swap(unique&);
  491. // destroy is called whenever the last copy of a multi_pass is
  492. // destructed (ownership_policy::release() returned true)
  493. //
  494. // mp: is a reference to the whole multi_pass instance
  495. template <typename MultiPass>
  496. static void destroy(MultiPass& mp);
  497. // docheck is called before the multi_pass is dereferenced or
  498. // incremented.
  499. //
  500. // mp: is a reference to the whole multi_pass instance
  501. //
  502. // This method is expected to make sure the multi_pass instance is
  503. // still valid. If it is invalid an exception should be thrown.
  504. template <typename MultiPass>
  505. static void docheck(MultiPass const& mp);
  506. // clear_queue is called whenever the function
  507. // multi_pass::clear_queue is called on this instance
  508. //
  509. // mp: is a reference to the whole multi_pass instance
  510. template <typename MultiPass>
  511. static void clear_queue(MultiPass& mp);
  512. };
  513. struct shared {};
  514. };
  515. It is possible to derive the struct `unique` from the type
  516. `boost::spirit::detail::default_checking_policy`. This type implements a
  517. minimal sufficient interface for some of the required functions, simplifying
  518. the task of writing a new checking policy.
  519. This class may implement a function `destroy()` being called during destruction
  520. of the last copy of a `multi_pass`. This function should be used to free any of
  521. the shared data items the policy might have allocated during construction of
  522. its `shared` part. Because of the way `multi_pass` is implemented any allocated
  523. data members in `shared` should _not_ be deep copied in a copy constructor of
  524. `shared`.
  525. [heading StoragePolicy]
  526. A `StoragePolicy` must have the following interface:
  527. struct storage_policy
  528. {
  529. // Value is the same type as typename MultiPass::value_type
  530. template <typename Value>
  531. struct unique
  532. {
  533. // destroy is called whenever the last copy of a multi_pass is
  534. // destructed (ownership_policy::release() returned true)
  535. //
  536. // mp: is a reference to the whole multi_pass instance
  537. template <typename MultiPass>
  538. static void destroy(MultiPass& mp);
  539. // swap is called by multi_pass::swap()
  540. void swap(unique&);
  541. // dereference is called whenever multi_pass::operator*() is invoked
  542. //
  543. // mp: is a reference to the whole multi_pass instance
  544. //
  545. // This function is expected to return a reference to the current
  546. // character/token.
  547. template <typename MultiPass>
  548. static typename MultiPass::reference dereference(MultiPass const& mp);
  549. // increment is called whenever multi_pass::operator++ is invoked
  550. //
  551. // mp: is a reference to the whole multi_pass instance
  552. template <typename MultiPass>
  553. static void increment(MultiPass& mp);
  554. //
  555. // mp: is a reference to the whole multi_pass instance
  556. template <typename MultiPass>
  557. static void clear_queue(MultiPass& mp);
  558. // is_eof is called to test whether this instance is a end of input
  559. // iterator.
  560. //
  561. // mp: is a reference to the whole multi_pass instance
  562. //
  563. // This method is expected to return true if the end of input is
  564. // reached.
  565. template <typename MultiPass>
  566. static bool is_eof(MultiPass const& mp);
  567. // less_than is called whenever multi_pass::operator==() is invoked
  568. //
  569. // mp: is a reference to the whole multi_pass instance
  570. // rhs: is the multi_pass reference this instance is compared
  571. // to
  572. //
  573. // This function is expected to return true if the current instance
  574. // is equal to the right hand side multi_pass instance
  575. template <typename MultiPass>
  576. static bool equal_to(MultiPass const& mp, MultiPass const& rhs);
  577. // less_than is called whenever multi_pass::operator<() is invoked
  578. //
  579. // mp: is a reference to the whole multi_pass instance
  580. // rhs: is the multi_pass reference this instance is compared
  581. // to
  582. //
  583. // This function is expected to return true if the current instance
  584. // is less than the right hand side multi_pass instance
  585. template <typename MultiPass>
  586. static bool less_than(MultiPass const& mp, MultiPass const& rhs);
  587. };
  588. // Value is the same type as typename MultiPass::value_type
  589. template <typename Value>
  590. struct shared {};
  591. };
  592. It is possible to derive the struct `unique` from the type
  593. `boost::spirit::detail::default_storage_policy`. This type implements a
  594. minimal sufficient interface for some of the required functions, simplifying
  595. the task of writing a new storage policy.
  596. This class may implement a function `destroy()` being called during destruction
  597. of the last copy of a `multi_pass`. This function should be used to free any of
  598. the shared data items the policy might have allocated during construction of
  599. its `shared` part. Because of the way `multi_pass` is implemented any allocated
  600. data members in `shared` should _not_ be deep copied in a copy constructor of
  601. `shared`.
  602. Generally, a `StoragePolicy` is the trickiest policy to implement. You should
  603. study and understand the existing `StoragePolicy` classes before you try and
  604. write your own.
  605. [endsect]