advanced_scan.h File Reference

More advanced text processing for splitting text into lines or tokens. More...

#include <vector>
#include "token.h"
#include "scan.h"
#include "lib_begin.h"
#include <cgv/config/lib_end.h>

Go to the source code of this file.

Classes
struct	cgv::utils::line
	a line in a text is simply represented as a token More...
struct	cgv::utils::typed_token
	a typed token also stores the type and value of a parsed token. More...

Namespaces
namespace	cgv
	the cgv namespace
namespace	cgv::utils
	namespace that holds tools that dont fit any other namespace

Enumerations
enum	cgv::utils::token_type {   PLAIN , URL , FLOAT_VALUE , TIME_VALUE ,   DATE_VALUE }
	different types that a typed_token can have More...

Functions
void	cgv::utils::split_to_tokens (const char *begin, const char *end, std::vector< token > &tokens, const std::string &separators, bool merge_separators=true, const std::string &open_parenthesis="", const std::string &close_parenthesis="", const std::string &whitespaces=" \t\n", unsigned int max_nr_tokens=-1)
	this function splits a text range into tokens.
void	cgv::utils::split_to_tokens (const token &tok, std::vector< token > &tokens, const std::string &separators, bool merge_separators=true, const std::string &open_parenthesis="", const std::string &close_parenthesis="", const std::string &whitespaces=" \t\n", unsigned int=-1)
	text range given as token
void	cgv::utils::split_to_tokens (const std::string &s, std::vector< token > &tokens, const std::string &separators, bool merge_separators=true, const std::string &open_parenthesis="", const std::string &close_parenthesis="", const std::string &whitespaces=" \t\n", unsigned int=(unsigned int) -1)
	text range given as string
void	cgv::utils::split_to_lines (const char *begin, const char *end, std::vector< line > &lines, bool truncate_trailing_spaces=true)
	this function splits a text range at the newline characters into single lines.
void	cgv::utils::split_to_lines (const token &tok, std::vector< line > &lines, bool truncate_trailing_spaces=true)
	text range given as token
void	cgv::utils::split_to_lines (const std::string &s, std::vector< line > &lines, bool truncate_trailing_spaces=true)
	text range given as string
bool	cgv::utils::balanced_find_content (const char *begin, const char *end, token &content, char open_parenthesis, char close_parenthesis)
	the input range must begin with an open parenthesis.
bool	cgv::utils::balanced_find_content (const token &expression, token &content, char open_parenthesis, char close_parenthesis)
bool	cgv::utils::balanced_find_content (const std::string &expression, token &content, char open_parenthesis, char close_parenthesis)
std::string	cgv::utils::strip_cpp_comments (const std::string &source, bool correct_new_lines=true)
	remove cpp-style comments from string

Detailed Description

More advanced text processing for splitting text into lines or tokens.

Definition in file advanced_scan.h.