Function Fast_custom_index->tokenise_text
Definitions
sources/database_search.php
- Tokenise some text, so it can be indexed by token.
- Visibility: protected
- Is abstract?: No
- Is static?: No
- Is final?: No
- Return: array
Parameters
| Name | Type | Default | Set | Range | Description |
|---|---|---|---|---|---|
| $text | string | required parameter | N/A | N/A | The text |
| $lang | LANGUAGE_NAME | required parameter | N/A | N/A | Language codename |
| $ngrams_exclude | ?array | Null | N/A | N/A | A list of ngrams to explicitly exclude (used internally to stop repetitions across multiple APPEARANCE_CONTEXTs, ultimately required to stop row repetition in output) (null: none) |
| $total_singular_ngram_tokens | ?integer | Null | N/A | N/A | Maintain a count of singular ngrams (typically words) in here (null: do not maintain) |
| $statistics_map | ?array | Null | N/A | N/A | Write into this map of singular ngram (typically, words) to number of occurrences (null: do not maintain a map) |
Return
- Map between ngrams and number of occurrences
- Type: array
- Set: N/A
- Range: N/A