Create a symbolic link.

    This will work on linux or windows, however windows does have some corner
    cases. For more details see notes in `ubelt._win32_links`.

    Args:
        path (PathLike): path to real file or directory
        link_path (PathLike): path to desired location for symlink
        overwrite (bool): overwrite existing symlinks.
            This will not overwrite real files on systems with proper symlinks.
            However, on older versions of windows junctions are
            indistinguishable from real files, so we cannot make this
            guarantee.  (default = False)
        verbose (int):  verbosity level (default=0)

    Returns:
        PathLike: link path

    CommandLine:
        python -m ubelt.util_links symlink:0

    Example:
        >>> import ubelt as ub
        >>> dpath = ub.ensure_app_cache_dir('ubelt', 'test_symlink0')
        >>> real_path = join(dpath, 'real_file.txt')
        >>> link_path = join(dpath, 'link_file.txt')
        >>> [ub.delete(p) for p in [real_path, link_path]]
        >>> ub.writeto(real_path, 'foo')
        >>> result = symlink(real_path, link_path)
        >>> assert ub.readfrom(result) == 'foo'
        >>> [ub.delete(p) for p in [real_path, link_path]]

    Example:
        >>> import ubelt as ub
        >>> from os.path import dirname
        >>> dpath = ub.ensure_app_cache_dir('ubelt', 'test_symlink1')
        >>> ub.delete(dpath)
        >>> ub.ensuredir(dpath)
        >>> _dirstats(dpath)
        >>> real_dpath = ub.ensuredir((dpath, 'real_dpath'))
        >>> link_dpath = ub.augpath(real_dpath, base='link_dpath')
        >>> real_path = join(dpath, 'afile.txt')
        >>> link_path = join(dpath, 'afile.txt')
        >>> [ub.delete(p) for p in [real_path, link_path]]
        >>> ub.writeto(real_path, 'foo')
        >>> result = symlink(real_dpath, link_dpath)
        >>> assert ub.readfrom(link_path) == 'foo', 'read should be same'
        >>> ub.writeto(link_path, 'bar')
        >>> _dirstats(dpath)
        >>> assert ub.readfrom(link_path) == 'bar', 'very bad bar'
        >>> assert ub.readfrom(real_path) == 'bar', 'changing link did not change real'
        >>> ub.writeto(real_path, 'baz')
        >>> _dirstats(dpath)
        >>> assert ub.readfrom(real_path) == 'baz', 'very bad baz'
        >>> assert ub.readfrom(link_path) == 'baz', 'changing real did not change link'
        >>> ub.delete(link_dpath, verbose=1)
        >>> _dirstats(dpath)
        >>> assert not exists(link_dpath), 'link should not exist'
        >>> assert exists(real_path), 'real path should exist'
        >>> _dirstats(dpath)
        >>> ub.delete(dpath, verbose=1)
        >>> _dirstats(dpath)
        >>> assert not exists(real_path)


Transforms function args into a key that can be used by the cache

    CommandLine:
        xdoctest -m ubelt.util_memoize _make_signature_key

    Example:
        >>> args = (4, [1, 2])
        >>> kwargs = {'a': 'b'}
        >>> key = _make_signature_key(args, kwargs)
        >>> print('key = {!r}'.format(key))
        >>> # Some mutable types cannot be handled by ub.hash_data
        >>> import pytest
        >>> import six
        >>> if six.PY2:
        >>>     import collections as abc
        >>> else:
        >>>     from collections import abc
        >>> with pytest.raises(TypeError):
        >>>     _make_signature_key((4, [1, 2], {1: 2, 'a': 'b'}), kwargs={})
        >>> class Dummy(abc.MutableSet):
        >>>     def __contains__(self, item): return None
        >>>     def __iter__(self): return iter([])
        >>>     def __len__(self): return 0
        >>>     def add(self, item, loc): return None
        >>>     def discard(self, item): return None
        >>> with pytest.raises(TypeError):
        >>>     _make_signature_key((Dummy(),), kwargs={})


r"""
    Colorizes text a single color using ansii tags.

    Args:
        text (str): text to colorize
        color (str): may be one of the following: yellow, blink, lightgray,
            underline, darkyellow, blue, darkblue, faint, fuchsia, black,
            white, red, brown, turquoise, bold, darkred, darkgreen, reset,
            standout, darkteal, darkgray, overline, purple, green, teal, fuscia

    Returns:
        str: text : colorized text.
            If pygments is not installed plain text is returned.

    CommandLine:
        python -c "import pygments.console; print(sorted(pygments.console.codes.keys()))"
        python -m ubelt.util_colors color_text

    Example:
        >>> text = 'raw text'
        >>> import pytest
        >>> import ubelt as ub
        >>> if ub.modname_to_modpath('pygments'):
        >>>     # Colors text only if pygments is installed
        >>>     assert color_text(text, 'red') == '\x1b[31;01mraw text\x1b[39;49;00m'
        >>>     assert color_text(text, None) == 'raw text'
        >>> else:
        >>>     # Otherwise text passes through unchanged
        >>>     assert color_text(text, 'red') == 'raw text'
        >>>     assert color_text(text, None) == 'raw text'


Generates unique items in the order they appear.

    Args:
        items (Iterable): list of items

        key (Callable, optional): custom normalization function.
            If specified returns items where `key(item)` is unique.

    Yields:
        object: a unique item from the input sequence

    CommandLine:
        python -m utool.util_list --exec-unique_ordered

    Example:
        >>> import ubelt as ub
        >>> items = [4, 6, 6, 0, 6, 1, 0, 2, 2, 1]
        >>> unique_items = list(ub.unique(items))
        >>> assert unique_items == [4, 6, 0, 1, 2]

    Example:
        >>> import ubelt as ub
        >>> items = ['A', 'a', 'b', 'B', 'C', 'c', 'D', 'e', 'D', 'E']
        >>> unique_items = list(ub.unique(items, key=six.text_type.lower))
        >>> assert unique_items == ['A', 'b', 'C', 'D', 'e']
        >>> unique_items = list(ub.unique(items))
        >>> assert unique_items == ['A', 'a', 'b', 'B', 'C', 'c', 'D', 'e', 'E']


Returns indices corresponding to the first instance of each unique item.

    Args:
        items (Sequence): indexable collection of items

        key (Callable, optional): custom normalization function.
            If specified returns items where `key(item)` is unique.

    Yields:
        int : indices of the unique items

    Example:
        >>> items = [0, 2, 5, 1, 1, 0, 2, 4]
        >>> indices = list(argunique(items))
        >>> assert indices == [0, 1, 2, 3, 7]
        >>> indices = list(argunique(items, key=lambda x: x % 2 == 0))
        >>> assert indices == [0, 2]


Returns a list of booleans corresponding to the first instance of each
    unique item.

    Args:
        items (Sequence): indexable collection of items

        key (Callable, optional): custom normalization function.
            If specified returns items where `key(item)` is unique.

    Returns:
        List[bool] : flags the items that are unique

    Example:
        >>> import ubelt as ub
        >>> items = [0, 2, 1, 1, 0, 9, 2]
        >>> flags = unique_flags(items)
        >>> assert flags == [True, True, True, False, False, True, False]
        >>> flags = unique_flags(items, key=lambda x: x % 2 == 0)
        >>> assert flags == [True, False, True, False, False, False, False]


Constructs a list of booleans where an item is True if its position is in
    `indices` otherwise it is False.

    Args:
        indices (list): list of integer indices

        maxval (int): length of the returned list. If not specified
            this is inferred from `indices`

    Note:
        In the future the arg `maxval` may change its name to `shape`

    Returns:
        list: mask: list of booleans. mask[idx] is True if idx in indices

    Example:
        >>> import ubelt as ub
        >>> indices = [0, 1, 4]
        >>> mask = ub.boolmask(indices, maxval=6)
        >>> assert mask == [True, True, False, False, True, False]
        >>> mask = ub.boolmask(indices)
        >>> assert mask == [True, True, False, False, True]


Determine if all items in a sequence are the same

    Args:
        iterable (Iterable): items to determine if they are all the same

        eq (Callable, optional): function to determine equality
            (default: operator.eq)

    Example:
        >>> allsame([1, 1, 1, 1])
        True
        >>> allsame([])
        True
        >>> allsame([0, 1])
        False
        >>> iterable = iter([0, 1, 1, 1])
        >>> next(iterable)
        >>> allsame(iterable)
        True
        >>> allsame(range(10))
        False
        >>> allsame(range(10), lambda a, b: True)
        True


Returns the indices that would sort a indexable object.

    This is similar to `numpy.argsort`, but it is written in pure python and
    works on both lists and dictionaries.

    Args:
        indexable (Iterable or Mapping): indexable to sort by

        key (Callable, optional): customizes the ordering of the indexable

        reverse (bool, optional): if True returns in descending order

    Returns:
        list: indices: list of indices such that sorts the indexable

    Example:
        >>> import ubelt as ub
        >>> # argsort works on dicts by returning keys
        >>> dict_ = {'a': 3, 'b': 2, 'c': 100}
        >>> indices = ub.argsort(dict_)
        >>> assert list(ub.take(dict_, indices)) == sorted(dict_.values())
        >>> # argsort works on lists by returning indices
        >>> indexable = [100, 2, 432, 10]
        >>> indices = ub.argsort(indexable)
        >>> assert list(ub.take(indexable, indices)) == sorted(indexable)
        >>> # Can use iterators, but be careful. It exhausts them.
        >>> indexable = reversed(range(100))
        >>> indices = ub.argsort(indexable)
        >>> assert indices[0] == 99
        >>> # Can use key just like sorted
        >>> indexable = [[0, 1, 2], [3, 4], [5]]
        >>> indices = ub.argsort(indexable, key=len)
        >>> assert indices == [2, 1, 0]
        >>> # Can use reverse just like sorted
        >>> indexable = [0, 2, 1]
        >>> indices = ub.argsort(indexable, reverse=True)
        >>> assert indices == [1, 2, 0]


Zips elementwise pairs between items1 and items2 into a dictionary. Values
    from items2 can be broadcast onto items1.

    Args:
        items1 (Iterable): full sequence
        items2 (Iterable): can either be a sequence of one item or a sequence
            of equal length to `items1`
        cls (Type[dict]): dictionary type to use. Defaults to dict, but could
            be ordered dict instead.

    Returns:
        dict: similar to dict(zip(items1, items2))

    Example:
        >>> assert dzip([1, 2, 3], [4]) == {1: 4, 2: 4, 3: 4}
        >>> assert dzip([1, 2, 3], [4, 4, 4]) == {1: 4, 2: 4, 3: 4}
        >>> assert dzip([], [4]) == {}


r"""
    Groups a list of items by group id.

    Args:
        items (Iterable): a list of items to group
        groupids (Iterable or Callable): a corresponding list of item groupids
            or a function mapping an item to a groupid.

    Returns:
        dict: groupid_to_items: maps a groupid to a list of items

    CommandLine:
        python -m ubelt.util_dict group_items

    Example:
        >>> import ubelt as ub
        >>> items    = ['ham',     'jam',   'spam',     'eggs',    'cheese', 'banana']
        >>> groupids = ['protein', 'fruit', 'protein',  'protein', 'dairy',  'fruit']
        >>> groupid_to_items = ub.group_items(items, groupids)
        >>> print(ub.repr2(groupid_to_items, nl=0))
        {'dairy': ['cheese'], 'fruit': ['jam', 'banana'], 'protein': ['ham', 'spam', 'eggs']}


Builds a histogram of items, counting the number of time each item appears
    in the input.

    Args:
        item_list (Iterable): hashable items (usually containing duplicates)
        weight_list (Iterable): corresponding weights for each item
        ordered (bool): if True the result is ordered by frequency
        labels (Iterable, optional): expected labels (default None)
            Allows this function to pre-initialize the histogram.
            If specified the frequency of each label is initialized to
            zero and item_list can only contain items specified in labels.

    Returns:
        dict : dictionary where the keys are items in item_list, and the values
          are the number of times the item appears in item_list.

    CommandLine:
        python -m ubelt.util_dict dict_hist

    Example:
        >>> import ubelt as ub
        >>> item_list = [1, 2, 39, 900, 1232, 900, 1232, 2, 2, 2, 900]
        >>> hist = ub.dict_hist(item_list)
        >>> print(ub.repr2(hist, nl=0))
        {1: 1, 2: 4, 39: 1, 900: 3, 1232: 2}

    Example:
        >>> import ubelt as ub
        >>> item_list = [1, 2, 39, 900, 1232, 900, 1232, 2, 2, 2, 900]
        >>> hist1 = ub.dict_hist(item_list)
        >>> hist2 = ub.dict_hist(item_list, ordered=True)
        >>> try:
        >>>     hist3 = ub.dict_hist(item_list, labels=[])
        >>> except KeyError:
        >>>     pass
        >>> else:
        >>>     raise AssertionError('expected key error')
        >>> #result = ub.repr2(hist_)
        >>> weight_list = [1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1]
        >>> hist4 = ub.dict_hist(item_list, weight_list=weight_list)
        >>> print(ub.repr2(hist1, nl=0))
        {1: 1, 2: 4, 39: 1, 900: 3, 1232: 2}
        >>> print(ub.repr2(hist4, nl=0))
        {1: 1, 2: 4, 39: 1, 900: 1, 1232: 0}


Find all duplicate items in a list.

    Search for all items that appear more than `k` times and return a mapping
    from each (k)-duplicate item to the positions it appeared in.

    Args:
        items (Iterable): hashable items possibly containing duplicates
        k (int): only return items that appear at least `k` times (default=2)
        key (Callable, optional): Returns indices where `key(items[i])`
            maps to a particular value at least k times.

    Returns:
        dict: maps each duplicate item to the indices at which it appears

    CommandLine:
        python -m ubelt.util_dict find_duplicates

    Example:
        >>> import ubelt as ub
        >>> items = [0, 0, 1, 2, 3, 3, 0, 12, 2, 9]
        >>> duplicates = ub.find_duplicates(items)
        >>> print('items = %r' % (items,))
        >>> print('duplicates = %r' % (duplicates,))
        >>> assert duplicates == {0: [0, 1, 6], 2: [3, 8], 3: [4, 5]}
        >>> assert ub.find_duplicates(items, 3) == {0: [0, 1, 6]}

    Example:
        >>> import ubelt as ub
        >>> items = [0, 0, 1, 2, 3, 3, 0, 12, 2, 9]
        >>> # note: k can be 0
        >>> duplicates = ub.find_duplicates(items, k=0)
        >>> print(ub.repr2(duplicates, nl=0))
        {0: [0, 1, 6], 1: [2], 2: [3, 8], 3: [4, 5], 9: [9], 12: [7]}

    Example:
        >>> import ubelt as ub
        >>> items = [10, 11, 12, 13, 14, 15, 16]
        >>> duplicates = ub.find_duplicates(items, key=lambda x: x // 2)
        >>> print(ub.repr2(duplicates, nl=0))
        {5: [0, 1], 6: [2, 3], 7: [4, 5]}


Constructs a dictionary that contains keys common between all inputs.
    The returned values will only belong to the first dictionary.

    Args:
        *args : a sequence of dictionaries (or sets of keys)

    Returns:
        Dict | OrderedDict :
            OrderedDict if the first argument is an OrderedDict, otherwise dict

    Notes:
        This function can be used as an alternative to `dict_subset` where any
        key not in the dictionary is ignored. See the following example:

        >>> dict_isect({'a': 1, 'b': 2, 'c': 3}, ['a', 'c', 'd'])
        {'a': 1, 'c': 3}

    Example:
        >>> dict_isect({'a': 1, 'b': 1}, {'b': 2, 'c': 2})
        {'b': 1}
        >>> dict_isect(odict([('a', 1), ('b', 2)]), odict([('c', 3)]))
        OrderedDict()
        >>> dict_isect()
        {}


applies a function to each of the keys in a dictionary

    Args:
        func (callable): a function or indexable object
        dict_ (dict): a dictionary

    Returns:
        newdict: transformed dictionary

    CommandLine:
        python -m ubelt.util_dict map_vals

    Example:
        >>> import ubelt as ub
        >>> dict_ = {'a': [1, 2, 3], 'b': []}
        >>> func = len
        >>> newdict = ub.map_vals(func, dict_)
        >>> assert newdict ==  {'a': 3, 'b': 0}
        >>> print(newdict)
        >>> # Can also use indexables as `func`
        >>> dict_ = {'a': 0, 'b': 1}
        >>> func = [42, 21]
        >>> newdict = ub.map_vals(func, dict_)
        >>> assert newdict ==  {'a': 42, 'b': 21}
        >>> print(newdict)


r"""
    Swaps the keys and values in a dictionary.

    Args:
        dict_ (dict): dictionary to invert
        unique_vals (bool): if False, inverted keys are returned in a set.
            The default is True.

    Returns:
        dict: inverted

    Notes:
        The must values be hashable.

        If the original dictionary contains duplicate values, then only one of
        the corresponding keys will be returned and the others will be
        discarded.  This can be prevented by setting `unique_vals=True`,
        causing the inverted keys to be returned in a set.

    CommandLine:
        python -m ubelt.util_dict invert_dict

    Example:
        >>> import ubelt as ub
        >>> dict_ = {'a': 1, 'b': 2}
        >>> inverted = ub.invert_dict(dict_)
        >>> assert inverted == {1: 'a', 2: 'b'}

    Example:
        >>> import ubelt as ub
        >>> dict_ = ub.odict([(2, 'a'), (1, 'b'), (0, 'c'), (None, 'd')])
        >>> inverted = ub.invert_dict(dict_)
        >>> assert list(inverted.keys())[0] == 'a'

    Example:
        >>> import ubelt as ub
        >>> dict_ = {'a': 1, 'b': 0, 'c': 0, 'd': 0, 'f': 2}
        >>> inverted = ub.invert_dict(dict_, unique_vals=False)
        >>> assert inverted == {0: {'b', 'c', 'd'}, 1: {'a'}, 2: {'f'}}


Recursively casts a AutoDict into a regular dictionary. All nested
        AutoDict values are also converted.

        Returns:
            dict: a copy of this dict without autovivification

        Example:
            >>> from ubelt.util_dict import AutoDict
            >>> auto = AutoDict()
            >>> auto[1] = 1
            >>> auto['n1'] = AutoDict()
            >>> static = auto.to_dict()
            >>> assert not isinstance(static, AutoDict)
            >>> assert not isinstance(static['n1'], AutoDict)


Perform a real symbolic link if possible. However, on most versions of
    windows you need special privledges to create a real symlink. Therefore, we
    try to create a symlink, but if that fails we fallback to using a junction.

    AFAIK, the main difference between symlinks and junctions are that symlinks
    can reference relative or absolute paths, where as junctions always
    reference absolute paths. Not 100% on this though. Windows is weird.

    Note that junctions will not register as links via `islink`, but I
    believe real symlinks will.


Creates real symlink. This will only work in versions greater than Windows
    Vista. Creating real symlinks requires admin permissions or at least
    specially enabled symlink permissions. On Windows 10 enabling developer
    mode should give you these permissions.


Determines if a path is a win32 junction

    CommandLine:
        python -m ubelt._win32_links _win32_is_junction

    Example:
        >>> # xdoc: +REQUIRES(WIN32)
        >>> import ubelt as ub
        >>> root = ub.ensure_app_cache_dir('ubelt', 'win32_junction')
        >>> ub.delete(root)
        >>> ub.ensuredir(root)
        >>> dpath = join(root, 'dpath')
        >>> djunc = join(root, 'djunc')
        >>> ub.ensuredir(dpath)
        >>> _win32_junction(dpath, djunc)
        >>> assert _win32_is_junction(djunc) is True
        >>> assert _win32_is_junction(dpath) is False
        >>> assert _win32_is_junction('notafile') is False


Returns the location that the junction points, raises ValueError if path is
    not a junction.

    CommandLine:
        python -m ubelt._win32_links _win32_read_junction

    Example:
        >>> # xdoc: +REQUIRES(WIN32)
        >>> import ubelt as ub
        >>> root = ub.ensure_app_cache_dir('ubelt', 'win32_junction')
        >>> ub.delete(root)
        >>> ub.ensuredir(root)
        >>> dpath = join(root, 'dpath')
        >>> djunc = join(root, 'djunc')
        >>> ub.ensuredir(dpath)
        >>> _win32_junction(dpath, djunc)
        >>> path = djunc
        >>> pointed = _win32_read_junction(path)
        >>> print('pointed = {!r}'.format(pointed))


rmtree for win32 that treats junctions like directory symlinks.
    The junction removal portion may not be safe on race conditions.

    There is a known issue that prevents shutil.rmtree from
    deleting directories with junctions.
    https://bugs.python.org/issue31226


Test if two hard links point to the same location

    CommandLine:
        python -m ubelt._win32_links _win32_is_hardlinked

    Example:
        >>> # xdoc: +REQUIRES(WIN32)
        >>> import ubelt as ub
        >>> root = ub.ensure_app_cache_dir('ubelt', 'win32_hardlink')
        >>> ub.delete(root)
        >>> ub.ensuredir(root)
        >>> fpath1 = join(root, 'fpath1')
        >>> fpath2 = join(root, 'fpath2')
        >>> ub.touch(fpath1)
        >>> ub.touch(fpath2)
        >>> fjunc1 = _win32_junction(fpath1, join(root, 'fjunc1'))
        >>> fjunc2 = _win32_junction(fpath2, join(root, 'fjunc2'))
        >>> assert _win32_is_hardlinked(fjunc1, fpath1)
        >>> assert _win32_is_hardlinked(fjunc2, fpath2)
        >>> assert not _win32_is_hardlinked(fjunc2, fpath1)
        >>> assert not _win32_is_hardlinked(fjunc1, fpath2)


Using the windows cmd shell to get information about a directory


Returns generators that double with each value returned
        Config includes optional start value


Retrieve the adjacency matrix from the nx.DiGraph or numpy array.


Apply causal discovery on observational data using CCDr.

        Args:
            data (pandas.DataFrame): DataFrame containing the data

        Returns:
            networkx.DiGraph: Solution given by the CCDR algorithm.


Save data to the csv format by default, in two separate files.

        Optional keyword arguments can be passed to pandas.


Launch an R script, starting from a template and replacing text in file
    before execution.

    Args:
        template (str): path to the template of the R script
        arguments (dict): Arguments that modify the template's placeholders
            with arguments
        output_function (function): Function to execute **after** the execution
            of the R script, and its output is returned by this function. Used
            traditionally as a function to retrieve the results of the
            execution.
        verbose (bool): Sets the verbosity of the R subprocess.
        debug (bool): If True, the generated scripts are not deleted.

    Return:
        Returns the output of the ``output_function`` if not `None`
        else `True` or `False` depending on whether the execution was
        successful.


Execute a subprocess to check the package's availability.

        Args:
            package (str): Name of the package to be tested.

        Returns:
            bool: `True` if the package is available, `False` otherwise


Perform the independence test.

        :param a: input data
        :param b: input data
        :type a: array-like, numerical data
        :type b: array-like, numerical data
        :return: dependency statistic (1=Highly dependent, 0=Not dependent)
        :rtype: float


Evaluate a graph taking account of the hardware.


Generate according to the topological order of the graph.


Use CGNN to create a graph from scratch. All the possible structures
        are tested, which leads to a super exponential complexity. It would be
        preferable to start from a graph skeleton for large graphs.

        Args:
            data (pandas.DataFrame): Observational data on which causal
               discovery has to be performed.
        Returns:
            networkx.DiGraph: Solution given by CGNN.


Modify and improve a directed acyclic graph solution using CGNN.

        Args:
            data (pandas.DataFrame): Observational data on which causal
               discovery has to be performed.
            dag (nx.DiGraph): Graph that provides the initial solution,
               on which the CGNN algorithm will be applied.
            alg (str): Exploration heuristic to use, among ["HC", "HCr",
               "tabu", "EHC"]
        Returns:
            networkx.DiGraph: Solution given by CGNN.


Orient the undirected graph using GNN and apply CGNN to improve the graph.

        Args:
            data (pandas.DataFrame): Observational data on which causal
               discovery has to be performed.
            umg (nx.Graph): Graph that provides the skeleton, on which the GNN
               then the CGNN algorithm will be applied.
            alg (str): Exploration heuristic to use, among ["HC", "HCr",
               "tabu", "EHC"]
        Returns:
            networkx.DiGraph: Solution given by CGNN.

        .. note::
           GNN (``cdt.causality.pairwise.GNN``) is first used to orient the
           undirected graph and output a DAG before applying CGNN.


Evaluate the entropy of the input variable.

    :param x: input variable 1D
    :return: entropy of x


Evaluate a pair using the IGCI model.

        :param a: Input variable 1D
        :param b: Input variable 1D
        :param kwargs: {refMeasure: Scaling method (gaussian, integral or None),
                        estimator: method used to evaluate the pairs (entropy or integral)}
        :return: Return value of the IGCI model >0 if a->b otherwise if return <0


Train the model.

        Args:
            x_tr (pd.DataFrame): CEPC format dataframe containing the pairs
            y_tr (pd.DataFrame or np.ndarray): labels associated to the pairs


Predict the causal score using a trained RCC model

        Args:
            x (numpy.array or pandas.DataFrame or pandas.Series): First variable or dataset.
            args (numpy.array): second variable (optional depending on the 1st argument).

        Returns:
            float: Causation score (Value : 1 if a->b and -1 if b->a)


For one variable, predict its neighbours.

        Args:
            df_features (pandas.DataFrame):
            df_target (pandas.Series):
            nh (int): number of hidden units
            idx (int): (optional) for printing purposes
            dropout (float): probability of dropout (between 0 and 1)
            activation_function (torch.nn.Module): activation function of the NN
            lr (float): learning rate of Adam
            l1 (float): L1 penalization coefficient
            batch_size (int): batch size, defaults to full-batch
            train_epochs (int): number of train epochs
            test_epochs (int): number of test epochs
            device (str): cuda or cpu device (defaults to ``cdt.SETTINGS.default_device``)
            verbose (bool): verbosity (defaults to ``cdt.SETTINGS.verbose``)
            nb_runs (int): number of bootstrap runs

        Returns:
            list: scores of each feature relatively to the target


Build a skeleton using a pairwise independence criterion.

        Args:
            data (pandas.DataFrame): Raw data table

        Returns:
            networkx.Graph: Undirected graph representing the skeleton.


Run GIES on an undirected graph.

        Args:
            data (pandas.DataFrame): DataFrame containing the data
            graph (networkx.Graph): Skeleton of the graph to orient

        Returns:
            networkx.DiGraph: Solution given by the GIES algorithm.


Feed-forward through the network.


Execute SAM on a dataset given a skeleton or not.

        Args:
            data (pandas.DataFrame): Observational data for estimation of causal relationships by SAM
            skeleton (numpy.ndarray): A priori knowledge about the causal relationships as an adjacency matrix.
                      Can be fed either directed or undirected links.
            nruns (int): Number of runs to be made for causal estimation.
                   Recommended: >=12 for optimal performance.
            njobs (int): Numbers of jobs to be run in Parallel.
                   Recommended: 1 if no GPU available, 2*number of GPUs else.
            gpus (int): Number of available GPUs for the algorithm.
            verbose (bool): verbose mode
            plot (bool): Plot losses interactively. Not recommended if nruns>1
            plot_generated_pair (bool): plots a generated pair interactively.  Not recommended if nruns>1
        Returns:
            networkx.DiGraph: Graph estimated by SAM, where A[i,j] is the term
            of the ith variable for the jth generator.


Infer causal relationships between 2 variables using the CDS statistic

        Args:
            a (numpy.ndarray): Variable 1
            b (numpy.ndarray): Variable 2

        Returns:
            float: Causation score (Value : 1 if a->b and -1 if b->a)


Prediction method for pairwise causal inference using the ANM model.

        Args:
            a (numpy.ndarray): Variable 1
            b (numpy.ndarray): Variable 2

        Returns:
            float: Causation score (Value : 1 if a->b and -1 if b->a)


Compute the fitness score of the ANM model in the x->y direction.

        Args:
            a (numpy.ndarray): Variable seen as cause
            b (numpy.ndarray): Variable seen as effect

        Returns:
            float: ANM fit score


Predict the graph skeleton.

        Args:
            data (pandas.DataFrame): observational data
            alpha (float): regularization parameter
            max_iter (int): maximum number of iterations

        Returns:
            networkx.Graph: Graph skeleton


Autoset GPU parameters using CUDA_VISIBLE_DEVICES variables.

    Return default config if variable not set.
    :param set_var: Variable to set. Must be of type ConfigSettings


Generic predict method, chooses which subfunction to use for a more
        suited.

        Depending on the type of `x` and of `*args`, this function process to execute
        different functions in the priority order:

        1. If ``args[0]`` is a ``networkx.(Di)Graph``, then ``self.orient_graph`` is executed.
        2. If ``args[0]`` exists, then ``self.predict_proba`` is executed.
        3. If ``x`` is a ``pandas.DataFrame``, then ``self.predict_dataset`` is executed.
        4. If ``x`` is a ``pandas.Series``, then ``self.predict_proba`` is executed.

        Args:
            x (numpy.array or pandas.DataFrame or pandas.Series): First variable or dataset.
            args (numpy.array or networkx.Graph): graph or second variable.

        Returns:
            pandas.Dataframe or networkx.Digraph: predictions output


Generic dataset prediction function.

        Runs the score independently on all pairs.

        Args:
            x (pandas.DataFrame): a CEPC format Dataframe.
            kwargs (dict): additional arguments for the algorithms

        Returns:
            pandas.DataFrame: a Dataframe with the predictions.


Run the algorithm on a directed_graph.

        Args:
            data (pandas.DataFrame): DataFrame containing the data
            graph (networkx.DiGraph): Skeleton of the graph to orient

        Returns:
            networkx.DiGraph: Solution on the given skeleton.

        .. warning::
           The algorithm is ran on the skeleton of the given graph.


Compute the gaussian kernel on a 1D vector.


Init a noise variable.


Runs Jarfo independently on all pairs.

        Args:
            x (pandas.DataFrame): a CEPC format Dataframe.
            kwargs (dict): additional arguments for the algorithms

        Returns:
            pandas.DataFrame: a Dataframe with the predictions.


Use Jarfo to predict the causal direction of a pair of vars.

        Args:
            a (numpy.ndarray): Variable 1
            b (numpy.ndarray): Variable 2
            idx (int): (optional) index number for printing purposes

        Returns:
            float: Causation score (Value : 1 if a->b and -1 if b->a)


Implementation of the ARACNE algorithm.

    Args:
     mat (numpy.ndarray): matrix, if it is a square matrix, the program assumes
         it is a relevance matrix where mat(i,j) represents the similarity content
         between nodes i and j. Elements of matrix should be
         non-negative.

    Returns:
    mat_nd (numpy.ndarray): Output deconvolved matrix (direct dependency matrix). Its components
        represent direct edge weights of observed interactions.

    .. note::
       Ref: ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context
       Adam A Margolin, Ilya Nemenman, Katia Basso, Chris Wiggins, Gustavo Stolovitzky, Riccardo Dalla Favera and Andrea Califano
       DOI: https://doi.org/10.1186/1471-2105-7-S1-S7


Apply deconvolution to a networkx graph.

    Args:
       g (networkx.Graph): Graph to apply deconvolution to
       alg (str): Algorithm to use ('aracne', 'clr', 'nd')
       kwargs (dict): extra options for algorithms

    Returns:
       networkx.Graph: graph with undirected links removed.


Input a graph and output a DAG.

    The heuristic is to reverse the edge with the lowest score of the cycle
    if possible, else remove it.

    Args:
        g (networkx.DiGraph): Graph to modify to output a DAG

    Returns:
        networkx.DiGraph: DAG made out of the input graph.


Returns the weighted average and standard deviation.

    values, weights -- numpy ndarrays with the same shape.


Pass data through the net structure.

        :param x: input data: shape (:,1)
        :type x: torch.Variable
        :return: output of the shallow net
        :rtype: torch.Variable


Run the GNN on a pair x,y of FloatTensor data.


Run multiple times GNN to estimate the causal direction.

        Args:
            a (np.ndarray): Variable 1
            b (np.ndarray): Variable 2
            nb_runs (int): number of runs to execute per batch (before testing for significance with t-test).
            nb_jobs (int): number of runs to execute in parallel. (Initialized with ``cdt.SETTINGS.NB_JOBS``)
            gpu (bool): use gpu (Initialized with ``cdt.SETTINGS.GPU``)
            idx (int): (optional) index of the pair, for printing purposes
            verbose (bool): verbosity (Initialized with ``cdt.SETTINGS.verbose``)
            ttest_threshold (float): threshold to stop the boostraps before ``nb_max_runs`` if the difference is significant
            nb_max_runs (int): Max number of bootstraps
            train_epochs (int): Number of epochs during which the model is going to be trained
            test_epochs (int): Number of epochs during which the model is going to be tested

        Returns:
            float: Causal score of the pair (Value : 1 if a->b and -1 if b->a)


Passing data through the network.

        :param x: 2d tensor containing both (x,y) Variables
        :return: output of the net


Updates all rows that match the filter.


Creates multiple new records in the database.

        This allows specifying custom conflict behavior using .on_conflict().
        If no special behavior was specified, this uses the normal Django create(..)

        Arguments:
            rows:
                An array of dictionaries, where each dictionary
                describes the fields to insert.

            return_model (default: False):
                If model instances should be returned rather than
                just dicts.

        Returns:
            A list of either the dicts of the rows inserted, including the pk or
            the models of the rows inserted with defaults for any fields not specified


Creates a new record in the database.

        This allows specifying custom conflict behavior using .on_conflict().
        If no special behavior was specified, this uses the normal Django create(..)

        Arguments:
            fields:
                The fields of the row to create.

        Returns:
            The primary key of the record that was created.


Creates a new record in the database and then gets
        the entire row.

        This allows specifying custom conflict behavior using .on_conflict().
        If no special behavior was specified, this uses the normal Django create(..)

        Arguments:
            fields:
                The fields of the row to create.

        Returns:
            The model instance representing the row that was created.


Verifies whether this field is gonna modify something
        on its own.

        "Magical" means that a field modifies the field value
        during the pre_save.

        Arguments:
            model_instance:
                The model instance the field is defined on.

            field:
                The field to get of whether the field is
                magical.

            is_insert:
                Pretend whether this is an insert?

        Returns:
            True when this field modifies something.


Gets the fields to use in an upsert.

        This some nice magic. We'll split the fields into
        a group of "insert fields" and "update fields":

        INSERT INTO bla ("val1", "val2") ON CONFLICT DO UPDATE SET val1 = EXCLUDED.val1

                         ^^^^^^^^^^^^^^                            ^^^^^^^^^^^^^^^^^^^^
                         insert_fields                                 update_fields

        Often, fields appear in both lists. But, for example,
        a :see:DateTime field with `auto_now_add=True` set, will
        only appear in "insert_fields", since it won't be set
        on existing rows.

        Other than that, the user specificies a list of fields
        in the upsert() call. That migt not be all fields. The
        user could decide to leave out optional fields. If we
        end up doing an update, we don't want to overwrite
        those non-specified fields.

        We cannot just take the list of fields the user
        specifies, because as mentioned, some fields
        make modifications to the model on their own.

        We'll have to detect which fields make modifications
        and include them in the list of insert/update fields.


When a model gets created or updated.


Compiles the HStore value into SQL.

        Compiles expressions contained in the values
        of HStore entries as well.

        Given a dictionary like:

            dict(key1='val1', key2='val2')

        The resulting SQL will be:

            hstore(hstore('key1', 'val1'), hstore('key2', 'val2'))


Adds an extra condition to an existing JOIN.

        This allows you to for example do:

            INNER JOIN othertable ON (mytable.id = othertable.other_id AND [extra conditions])

        This does not work if nothing else in your query doesn't already generate the
        initial join in the first place.


Sets the values to be used in this query.

        Insert fields are fields that are definitely
        going to be inserted, and if an existing row
        is found, are going to be overwritten with the
        specified value.

        Update fields are fields that should be overwritten
        in case an update takes place rather than an insert.
        If we're dealing with a INSERT, these will not be used.

        Arguments:
            objs:
                The objects to apply this query to.

            insert_fields:
                The fields to use in the INSERT statement

            update_fields:
                The fields to only use in the UPDATE statement.


Creates a REQUIRED CONSTRAINT for the specified hstore key.


Renames an existing REQUIRED CONSTRAINT for the specified
        hstore key.


Drops a REQUIRED CONSTRAINT for the specified hstore key.


Gets the name for a CONSTRAINT that applies
        to a single hstore key.

        Arguments:
            table:
                The name of the table the field is
                a part of.

            field:
                The hstore field to create a
                UNIQUE INDEX for.

            key:
                The name of the hstore key
                to create the name for.

        Returns:
            The name for the UNIQUE index.


Creates the actual SQL used when applying the migration.


Ran to prepare the configured database.

        This is where we enable the `hstore` extension
        if it wasn't enabled yet.


Override the base class so it doesn't cast all values
        to strings.

        psqlextra supports expressions in hstore fields, so casting
        all values to strings is a bad idea.


Rewrites a formed SQL INSERT query to include
        the ON CONFLICT clause.

        Arguments:
            sql:
                The SQL INSERT query to rewrite.

            params:
                The parameters passed to the query.

            returning:
                What to put in the `RETURNING` clause
                of the resulting query.

        Returns:
            A tuple of the rewritten SQL query and new params.


Rewrites a formed SQL INSERT query to include
        the ON CONFLICT DO UPDATE clause.


Rewrites a formed SQL INSERT query to include
        the ON CONFLICT DO NOTHING clause.


Builds the `conflict_target` for the ON CONFLICT
        clause.


Formats a field's name for usage in SQL.

        Arguments:
            field_name:
                The field name to format.

        Returns:
            The specified field name formatted for
            usage in SQL.


Formats a field's value for usage in SQL.

        Arguments:
            field_name:
                The name of the field to format
                the value of.

        Returns:
            The field's value formatted for usage
            in SQL.


Creates a UNIQUE constraint for the specified hstore keys.


Renames an existing UNIQUE constraint for the specified
        hstore keys.


Gets the name for a UNIQUE INDEX that applies
        to one or more keys in a hstore field.

        Arguments:
            table:
                The name of the table the field is
                a part of.

            field:
                The hstore field to create a
                UNIQUE INDEX for.

            key:
                The name of the hstore key
                to create the name for.

                This can also be a tuple
                of multiple names.

        Returns:
            The name for the UNIQUE index.


Iterates over the keys marked as "unique"
        in the specified field.

        Arguments:
            field:
                The field of which key's to
                iterate over.


Adds an extra condition to this join.

        Arguments:
            field:
                The field that the condition will apply to.

            value:
                The value to compare.


Compiles this JOIN into a SQL string.


Approximate the 95% confidence interval for Student's T distribution.

    Given the degrees of freedom, returns an approximation to the 95%
    confidence interval for the Student's T distribution.

    Args:
        df: An integer, the number of degrees of freedom.

    Returns:
        A float.


Find the pooled sample variance for two samples.

    Args:
        sample1: one sample.
        sample2: the other sample.

    Returns:
        Pooled sample variance, as a float.


Determine whether two samples differ significantly.

    This uses a Student's two-sample, two-tailed t-test with alpha=0.95.

    Args:
        sample1: one sample.
        sample2: the other sample.

    Returns:
        (significant, t_score) where significant is a bool indicating whether
        the two samples differ significantly; t_score is the score from the
        two-sample T test.


Return a topological sorting of nodes in a graph.

    roots - list of root nodes to search from
    getParents - function which returns the parents of a given node


N-Queens solver.

    Args:
        queen_count: the number of queens to solve for. This is also the
            board size.

    Yields:
        Solutions to the problem. Each yielded value is looks like
        (3, 8, 2, 1, 4, ..., 6) where each number is the column position for the
        queen, and the index into the tuple indicates the row.


uct tree search


random play until both players pass


Filters out benchmarks not supported by both Pythons.

    Args:
        benchmarks: a set() of benchmark names
        bench_funcs: dict mapping benchmark names to functions
        python: the interpereter commands (as lists)

    Returns:
        The filtered set of benchmark names


Recursively expand name benchmark names.

    Args:
        bm_name: string naming a benchmark or benchmark group.

    Yields:
        Names of actual benchmarks, with all group names fully expanded.


Initialize the strings we'll run the regexes against.

    The strings used in the benchmark are prefixed and suffixed by
    strings that are repeated n times.

    The sequence n_values contains the values for n.
    If n_values is None the values of n from the original benchmark
    are used.

    The generated list of strings is cached in the string_tables
    variable, which is indexed by n.

    Returns:
    A list of string prefix/suffix lengths.


Returns the domain of the B-Spline


Fetch the messages.

        :param category: the category of items to fetch
        :param kwargs: backend arguments

        :returns: a generator of items


Fetch the entries from the url.

        The method retrieves all entries from a RSS url

        :param category: the category of items to fetch

        :returns: a generator of entries


Fetch the entries

        :param category: the category of items to fetch
        :param kwargs: backend arguments

        :returns: a generator of items


Returns the RSS argument parser.


Fetch the bugs from the repository.

        The method retrieves, from a Bugzilla repository, the bugs
        updated since the given date.

        :param category: the category of items to fetch
        :param from_date: obtain bugs updated since this date

        :returns: a generator of bugs


Get issue notes


Get merge versions


Get the merge requests from pagination


Get the merge versions from pagination


Get the notes from pagination


Get emojis of a note


Initialize rate limit information


Returns the GitLab argument parser.


Fetch the messages from the channel.

        This method fetches the messages stored on the channel that were
        sent since the given date.

        :param category: the category of items to fetch
        :param from_date: obtain messages sent since this date

        :returns: a generator of messages


Fetch the number of members in a conversation, which is a supertype for public and
        private ones, DM and group DM.

        :param conversation: the ID of the conversation


Fetch user info.


Returns the Slack argument parser.


Extracts and coverts the update time from a Bugzilla item.

        The timestamp is extracted from 'delta_ts' field. This date is
        converted to UNIX timestamp format. Due Bugzilla servers ignore
        the timezone on HTTP requests, it will be ignored during the
        conversion, too.

        :param item: item generated by the backend

        :returns: a UNIX timestamp


Parse a Bugilla bugs details XML stream.

        This method returns a generator which parses the given XML,
        producing an iterator of dictionaries. Each dictionary stores
        the information related to a parsed bug.

        If the given XML is invalid or does not contains any bug, the
        method will raise a ParseError exception.

        :param raw_xml: XML string to parse

        :returns: a generator of parsed bugs

        :raises ParseError: raised when an error occurs parsing
            the given XML stream


Logout from the server.


Get metadata information in XML format.


Get a summary of bugs in CSV format.

        :param from_date: retrieve bugs that where updated from that date


Get the information of a list of bugs in XML format.

        :param bug_ids: list of bug identifiers


Get the activity of a bug in HTML format.

        :param bug_id: bug identifier


Fetch the events from the server.

        This method fetches those events of a group stored on the server
        that were updated since the given date. Data comments and rsvps
        are included within each event.

        :param category: the category of items to fetch
        :param from_date: obtain events updated since this date
        :param to_date: obtain events updated before this date
        :param filter_classified: remove classified fields from the resulting items

        :returns: a generator of events


Fetch the events

        :param category: the category of items to fetch
        :param kwargs: backend arguments

        :returns: a generator of items


Fetch the rsvps of a given event.


Fetch an Askbot HTML question body.

        The method fetchs the HTML question retrieving the
        question body of the item question received

        :param question: item with the question itself

        :returns: a list of HTML page/s for the question


Fetch all the comments of an Askbot question and answers.

        The method fetchs the list of every comment existing in a question and
        its answers.

        :param question: item with the question itself

        :returns: a list of comments with the ids as hashes


Build an Askbot HTML response.

        The method puts together all the information regarding a question

        :param html_question: array of HTML raw pages
        :param question: question object from the API
        :param comments: list of comments to add

        :returns: a dict item with the parsed question information


Retrieve a question page using the API.

        :param page: page to retrieve


Retrieve a raw HTML question and all it's information.

        :param question_id: question identifier
        :param page: page to retrieve


Retrieve a list of comments by a given id.

        :param object_id: object identifiere


Parse the question info container of a given HTML question.

        The method parses the information available in the question information
        container. The container can have up to 2 elements: the first one
        contains the information related with the user who generated the question
        and the date (if any). The second one contains the date of the updated,
        and the user who updated it (if not the same who generated the question).

        :param html_question: raw HTML question element

        :returns: an object with the parsed information


Parse the answers of a given HTML question.

        The method parses the answers related with a given HTML question,
        as well as all the comments related to the answer.

        :param html_question: raw HTML question element

        :returns: a list with the answers


Parse number of answer pages to paginate over them.

        :param html_question: raw HTML question element

        :returns: an integer with the number of pages


Parse the user information of a given HTML container.

        The method parses all the available user information in the container.
        If the class "user-info" exists, the method will get all the available
        information in the container. If not, if a class "tip" exists, it will be
        a wiki post with no user associated. Else, it can be an empty container.

        :param update_info: beautiful soup answer container element

        :returns: an object with the parsed information


Specific fetch for gerrit 2.8 version.

        Get open and closed reviews in different queries.
        Take the newer review from both lists and iterate.


Return the Gerrit server version.


Get the reviews starting from last_item.


Execute gerrit command against the archive


Execute gerrit command with retry if it fails


Get data associated to an issue


Get attachments of an issue


Get activities on an issue


Get data associated to an user


Get the user data by URL


Get the issue data by its ID


Get a collection list of a given issue


Build URL project


Fetch the groupsio paginated subscriptions for a given token

        :param per_page: number of subscriptions per page

        :returns: an iterator of subscriptions


Fetch requests from groupsio API


Generate a UUID based on the given parameters.

    The UUID will be the SHA1 of the concatenation of the values
    from the list. The separator bewteedn these values is ':'.
    Each value must be a non-empty string, otherwise, the function
    will raise an exception.

    :param *args: list of arguments used to generate the UUID

    :returns: a universal unique identifier

    :raises ValueError: when anyone of the values is not a string,
        is empty or `None`.


Fetch items from an archive manager.

    Generator to get the items of a category (previously fetched
    by the given backend class) from an archive manager. Only those
    items archived after the given date will be returned.

    The parameters needed to initialize `backend` and get the
    items are given using `backend_args` dict parameter.

    :param backend_class: backend class to retrive items
    :param backend_args: dict of arguments needed to retrieve the items
    :param manager: archive manager where the items will be retrieved
    :param category: category of the items to retrieve
    :param archived_after: return items archived after this date

    :returns: a generator of archived items


Find available backends.

    Look for the Perceval backends and commands under `top_package`
    and its sub-packages. When `top_package` defines a namespace,
    backends under that same namespace will be found too.

    :param top_package: package storing backends

    :returns: a tuple with two dicts: one with `Backend` classes and one
        with `BackendCommand` classes


Fetch items from the repository.

        The method retrieves items from a repository.

        To removed classified fields from the resulting items, set
        the parameter `filter_classified`. Take into account this
        parameter is incompatible with archiving items. Raw client
        data are archived before any other process. Therefore,
        classified data  are stored within the archive. To prevent
        from possible data leaks or security issues when users do
        not need these fields, archiving and filtering are not
        compatible.

        :param category: the category of the items fetched
        :param filter_classified: remove classified fields from the resulting items
        :param kwargs: a list of other parameters (e.g., from_date, offset, etc.
        specific for each backend)

        :returns: a generator of items

        :raises BackendError: either when the category is not valid or
            'filter_classified' and 'archive' are active at the same time.


Fetch the questions from an archive.

        It returns the items stored within an archive. If this method is called but
        no archive was provided, the method will raise a `ArchiveError` exception.

        :returns: a generator of items

        :raises ArchiveError: raised when an error occurs accessing an archive


Remove classified or confidential data from an item.

        It removes those fields that contain data considered as classified.
        Classified fields are defined in `CLASSIFIED_FIELDS` class attribute.

        :param item: fields will be removed from this item

        :returns: the same item but with confidential data filtered


Parse a list of arguments.

        Parse argument strings needed to run a backend command. The result
        will be a `argparse.Namespace` object populated with the values
        obtained after the validation of the parameters.

        :param args: argument strings

        :result: an object with the parsed values


Activate archive arguments parsing


Activate output arguments parsing


Fetch and write items.

        This method runs the backend to fetch the items from the given
        origin. Items are converted to JSON objects and written to the
        defined output.

        If `fetch-archive` parameter was given as an argument during
        the inizialization of the instance, the items will be retrieved
        using the archive manager.


Initialize archive based on the parsed parameters


Extracts the update time from a MBox item.

        The timestamp used is extracted from 'Date' field in its
        several forms. This date is converted to UNIX timestamp
        format.

        :param item: item generated by the backend

        :returns: a UNIX timestamp


Fetch and parse the messages from a mailing list


Copy the contents of a mbox to a temporary file


Fetch the commits

        :param category: the category of items to fetch
        :param kwargs: backend arguments

        :returns: a generator of items


Returns the Git argument parser.


Parse the Git log stream.


Clone a Git repository.

        Make a bare copy of the repository stored in `uri` into `dirpath`.
        The repository would be either local or remote.

        :param uri: URI of the repository
        :param dirtpath: directory where the repository will be cloned

        :returns: a `GitRepository` class having cloned the repository

        :raises RepositoryError: when an error occurs cloning the given
            repository


Count the objects of a repository.

        The method returns the total number of objects (packed and unpacked)
        available on the repository.

        :raises RepositoryError: when an error occurs counting the objects
            of a repository


Check if the repo is in a detached state.

        The repository is in a detached state when HEAD is not a symbolic
        reference.

        :returns: whether the repository is detached or not

        :raises RepositoryError: when an error occurs checking the state
            of the repository


Keep the repository in sync.

        This method will synchronize the repository with its 'origin',
        fetching newest objects and updating references. It uses low
        level commands which allow to keep track of which things
        have changed in the repository.

        The method also returns a list of hashes related to the new
        commits fetched during the process.

        :returns: list of new commits

        :raises RepositoryError: when an error occurs synchronizing
            the repository


Read the commit log from the repository.

        The method returns the Git log of the repository using the
        following options:

            git log --raw --numstat --pretty=fuller --decorate=full
                --all --reverse --topo-order --parents -M -C -c
                --remotes=origin

        When `from_date` is given, it gets the commits equal or older
        than that date. This date is given in a datetime object.

        The list of branches is a list of strings, with the names of the
        branches to fetch. If the list of branches is empty, no commit
        is fetched. If the list of branches is None, all commits
        for all branches will be fetched.

        :param from_date: fetch commits newer than a specific
            date (inclusive)
        :param branches: names of branches to fetch from (default: None)
        :param encoding: encode the log using this format

        :returns: a generator where each item is a line from the log

        :raises EmptyRepositoryError: when the repository is empty and
            the action cannot be performed
        :raises RepositoryError: when an error occurs fetching the log


Show the data of a set of commits.

        The method returns the output of Git show command for a
        set of commits using the following options:

            git show --raw --numstat --pretty=fuller --decorate=full
                --parents -M -C -c [<commit>...<commit>]

        When the list of commits is empty, the command will return
        data about the last commit, like the default behaviour of
        `git show`.

        :param commits: list of commits to show data
        :param encoding: encode the output using this format

        :returns: a generator where each item is a line from the show output

        :raises EmptyRepositoryError: when the repository is empty and
            the action cannot be performed
        :raises RepositoryError: when an error occurs fetching the show output


Update references removing old ones.


Get the current list of local or remote refs.


Reads self.proc.stderr.

        Usually, this should be read in a thread, to prevent blocking
        the read from stdout of the stderr buffer is filled, and this
        function is not called becuase the program is busy in the
        stderr reading loop.

        Reads self.proc.stderr (self.proc is the subprocess running
        the git command), and reads / writes self.failed_message
        (the message sent to stderr when git fails, usually one line).


Run a command.

        Execute `cmd` command in the directory set by `cwd`. Environment
        variables can be set using the `env` dictionary. The output
        data is returned as encoded bytes.

        Commands which their returning status codes are non-zero will
        be treated as failed. Error codes considered as valid can be
        ignored giving them in the `ignored_error_codes` list.

        :returns: the output of the command as encoded bytes

        :raises RepositoryError: when an error occurs running the command


Fetch the tweets from the server.

        This method fetches tweets from the TwitterSearch API published in the last seven days.

        :param category: the category of items to fetch
        :param since_id: if not null, it returns results with an ID greater than the specified ID
        :param max_id: when it is set or if not None, it returns results with an ID less than the specified ID
        :param geocode: if enabled, returns tweets by users located at latitude,longitude,"mi"|"km"
        :param lang: if enabled, restricts tweets to the given language, given by an ISO 639-1 code
        :param include_entities: if disabled, it excludes entities node
        :param tweets_type: type of tweets returned. Default is “mixed”, others are "recent" and "popular"

        :returns: a generator of tweets


Fetch the tweets

        :param category: the category of items to fetch
        :param kwargs: backend arguments

        :returns: a generator of items


Returns the Twitter argument parser.


Fetch data from Google API.

        The method retrieves a list of hits for some
        given keywords using the Google API.

        :param category: the category of items to fetch

        :returns: a generator of data


Fetch Google hit items

        :param category: the category of items to fetch
        :param kwargs: backend arguments

        :returns: a generator of items


Parse the hits returned by the Google Search API


Get repo info about stars, watchers and forks


Get issue reactions


Get reactions on issue comments


Get issue assignees


Get pull request requested reviewers


Get pull request commit hashes


Get pull review comment reactions


Get reactions of an issue


Fetch the issues from the repository.

        The method retrieves, from a GitHub repository, the issues
        updated since the given date.

        :param from_date: obtain issues updated since this date

        :returns: a generator of issues


Get pull requested reviewers


Get pull request commits


Get reactions of a review comment


Get the user information and update the user cache


Return array of all tokens remaining API points


Check if we need to switch GitHub API tokens


Update rate limits data for the current token


Init metadata information.

        Metatada is composed by basic information needed to identify
        where archived data came from and how it can be retrieved
        and built into Perceval items.

        :param: origin: identifier of the repository
        :param: backend_name: name of the backend
        :param: backend_version: version of the backend
        :param: category: category of the items fetched
        :param: backend_params: dict representation of the fetch parameters

        raises ArchiveError: when an error occurs initializing the metadata


Store a raw item in this archive.

        The method will store `data` content in this archive. The unique
        identifier for that item will be generated using the rest of the
        parameters.

        :param uri: request URI
        :param payload: request payload
        :param headers: request headers
        :param data: data to store in this archive

        :raises ArchiveError: when an error occurs storing the given data


Retrieve a raw item from the archive.

        The method will return the `data` content corresponding to the
        hascode derived from the given parameters.

        :param uri: request URI
        :param payload: request payload
        :param headers: request headers

        :returns: the archived data

        :raises ArchiveError: when an error occurs retrieving data


Create a brand new archive.

         Call this method to create a new and empty archive. It will initialize
         the storage file in the path defined by `archive_path`.

        :param archive_path: absolute path where the archive file will be created

        :raises ArchiveError: when the archive file already exists


Generate a SHA1 based on the given arguments.

        Hashcodes created by this method will used as unique identifiers
        for the raw items or resources stored by this archive.

        :param uri: URI to the resource
        :param payload: payload of the request needed to fetch the resource
        :param headers: headers of the request needed to fetch the resource

        :returns: a SHA1 hash code


Check whether the archive is valid or not.

        This method will check if tables were created and if they
        contain valid data.


Fetch the number of rows in a table


Remove an archive.

        This method deletes from the filesystem the archive stored
        in `archive_path`.

        :param archive_path: path to the archive

        :raises ArchiveManangerError: when an error occurs removing the
            archive


Search archives.

        Get the archives which store data based on the given parameters.
        These parameters define which the origin was (`origin`), how data
        was fetched (`backend_name`) and data type ('category').
        Only those archives created on or after `archived_after` will be
        returned.

        The method returns a list with the file paths to those archives.
        The list is sorted by the date of creation of each archive.

        :param origin: data origin
        :param backend_name: backed used to fetch data
        :param category: type of the items fetched by the backend
        :param archived_after: get archives created on or after this date

        :returns: a list with archive names which match the search criteria


Search archives using filters.


Check if filename is a compressed file supported by the tool.

    This function uses magic numbers (first four bytes) to determine
    the type of the file. Supported types are 'gz' and 'bz2'. When
    the filetype is not supported, the function returns `None`.

    :param filepath: path to the file

    :returns: 'gz' or 'bz2'; `None` if the type is not supported


Generate a months range.

    Generator of months starting on `from_date` util `to_date`. Each
    returned item is a tuple of two datatime objects like in (month, month+1).
    Thus, the result will follow the sequence:
        ((fd, fd+1), (fd+1, fd+2), ..., (td-2, td-1), (td-1, td))

    :param from_date: generate dates starting on this month
    :param to_date: generate dates until this month

    :result: a generator of months range


Convert an email message into a dictionary.

    This function transforms an `email.message.Message` object
    into a dictionary. Headers are stored as key:value pairs
    while the body of the message is stored inside `body` key.
    Body may have two other keys inside, 'plain', for plain body
    messages and 'html', for HTML encoded messages.

    The returned dictionary has the type `requests.structures.CaseInsensitiveDict`
    due to same headers with different case formats can appear in
    the same message.

    :param msg: email message of type `email.message.Message`

    :returns : dictionary of type `requests.structures.CaseInsensitiveDict`

    :raises ParseError: when an error occurs transforming the message
        to a dictionary


Remove control and invalid characters from an xml stream.

    Looks for invalid characters and subtitutes them with whitespaces.
    This solution is based on these two posts: Olemis Lang's reponse
    on StackOverflow (http://stackoverflow.com/questions/1707890) and
    lawlesst's on GitHub Gist (https://gist.github.com/lawlesst/4110923),
    that is based on the previous answer.

    :param xml: XML stream

    :returns: a purged XML stream


Convert a XML stream into a dictionary.

    This function transforms a xml stream into a dictionary. The
    attributes are stored as single elements while child nodes are
    stored into lists. The text node is stored using the special
    key '__text__'.

    This code is based on Winston Ewert's solution to this problem.
    See http://codereview.stackexchange.com/questions/10400/convert-elementtree-to-dict
    for more info. The code was licensed as cc by-sa 3.0.

    :param raw_xml: XML stream

    :returns: a dict with the XML data

    :raises ParseError: raised when an error occurs parsing the given
        XML stream


Parse a Redmine issues JSON stream.

        The method parses a JSON stream and returns a list iterator.
        Each item is a dictionary that contains the issue parsed data.

        :param raw_json: JSON string to parse

        :returns: a generator of parsed issues


Get the information of the given issue.

        :param issue_id: issue identifier


Get the information of the given user.

        :param user_id: user identifier


Call to get a resource.

        :param method: resource to get
        :param params: dict with the HTTP parameters needed to get
            the given resource


Fetch data from a Docker Hub repository.

        The method retrieves, from a repository stored in Docker Hub,
        its data which includes number of pulls, stars, description,
        among other data.

        :param category: the category of items to fetch

        :returns: a generator of data


Fetch the Dockher Hub items

        :param category: the category of items to fetch
        :param kwargs: backend arguments

        :returns: a generator of items


Add extra information for custom fields.

    :param custom_fields: set of custom fields with the extra information
    :param fields: fields of the issue where to add the extra information

    :returns: an set of items with the extra information mapped


Retrieve all the items from a given date.

        :param url: endpoint API url
        :param from_date: obtain items updated since this date
        :param expand_fields: if True, it includes the expand fields in the payload


Retrieve all the issues from a given date.

        :param from_date: obtain issues updated since this date


Retrieve all the comments of a given issue.

        :param issue_id: ID of the issue


Retrieve all the fields available.


Retrieve all the questions from a given date.

        :param from_date: obtain questions updated since this date


Returns the StackExchange argument parser.


Fetch the pages

        :param category: the category of items to fetch
        :param kwargs: backend arguments

        :returns: a generator of items


Get the max date in unixtime format from reviews.


Retrieve recent pages from all namespaces starting from rccontinue.


Fetch the messages the bot can read from the server.

        The method retrieves, from the Telegram server, the messages
        sent with an offset equal or greater than the given.

        A list of chats, groups and channels identifiers can be set
        using the parameter `chats`. When it is set, only those
        messages sent to any of these will be returned. An empty list
        will return no messages.

        :param category: the category of items to fetch
        :param offset: obtain messages from this offset
        :param chats: list of chat names used to filter messages

        :returns: a generator of messages

        :raises ValueError: when `chats` is an empty list


Check if a message can be filtered based in a list of chats.

        This method returns `True` when the message was sent to a chat
        of the given list. It also returns `True` when chats is `None`.

        :param message: Telegram message
        :param chats: list of chat, groups and channels identifiers

        :returns: `True` when the message can be filtered; otherwise,
            it returns `False`


Fetch the articles

        :param category: the category of items to fetch
        :param kwargs: backend arguments

        :returns: a generator of items


NNTP metadata.

        This method takes items, overriding `metadata` decorator,
        to add extra information related to NNTP.

        :param item: an item fetched by a backend
        :param filter_classified: sets if classified fields were filtered


Parse a NNTP article.

        This method parses a NNTP article stored in a string object
        and returns an dictionary.

        :param raw_article: NNTP article string

        :returns: a dictionary of type `requests.structures.CaseInsensitiveDict`

        :raises ParseError: when an error is found parsing the article


Fetch NNTP data from the server or from the archive

        :param method: the name of the command to execute
        :param args: the arguments required by the command


Fetch article data

        :param article_id: id of the article to fetch


Fetch data from NNTP

        :param method: the name of the command to execute
        :param args: the arguments required by the command


Fetch data from the archive

        :param method: the name of the command to execute
        :param args: the arguments required by the command


Create a http session and initialize the retry object.


The fetching process sleeps until the rate limit is restored or
           raises a RateLimitError exception if sleep_for_rate flag is disabled.


Parse a Supybot IRC stream.

        Returns an iterator of dicts. Each dicts contains information
        about the date, type, nick and body of a single log entry.

        :returns: iterator of parsed lines

        :raises ParseError: when an invalid line is found parsing the given
            stream


Parse timestamp section


Parse message section


Fetch the topics

        :param category: the category of items to fetch
        :param kwargs: backend arguments

        :returns: a generator of items


Parse a topics page stream.

        The result of parsing process is a generator of tuples. Each
        tuple contains de identifier of the topic, the last date
        when it was updated and whether is pinned or not.

        :param raw_json: JSON stream to parse

        :returns: a generator of parsed bugs


Retrieve the post whit `post_id` identifier.

        :param post_id: identifier of the post to retrieve


Fetch the tasks

        :param category: the category of items to fetch
        :param kwargs: backend arguments

        :returns: a generator of items


Parse a Phabricator tasks JSON stream.

        The method parses a JSON stream and returns a list iterator.
        Each item is a dictionary that contains the task parsed data.

        :param raw_json: JSON string to parse

        :returns: a generator of parsed tasks


Parse a Phabricator users JSON stream.

        The method parses a JSON stream and returns a list iterator.
        Each item is a dictionary that contais the user parsed data.

        :param raw_json: JSON string to parse

        :returns: a generator of parsed users


Retrieve tasks.

        :param from_date: retrieve tasks that where updated from that date;
            dates are converted epoch time.


Retrieve tasks transactions.

        :param phids: list of tasks identifiers


Extracts the identifier from a Confluence item.

        This identifier will be the mix of two fields because a
        historical content does not have any unique identifier.
        In this case, 'id' and 'version' values are combined because
        it should not be possible to have two equal version numbers
        for the same content. The value to return will follow the
        pattern: <content>#v<version> (i.e 28979#v10).


Parse the result property, extracting the value
        and unit of measure


Return a capabilities url


Get and parse a WFS capabilities document, returning an
        instance of WFSCapabilitiesInfoset

        Parameters
        ----------
        url : string
            The URL to the WFS capabilities document.
        timeout : number
            A timeout value (in seconds) for the request.


Parse a WFS capabilities document, returning an
        instance of WFSCapabilitiesInfoset

        string should be an XML capabilities document


helper function to build a WFS 3.0 URL

        @type path: string
        @param path: path of WFS URL

        @returns: fully constructed URL path


Consruct fiona schema based on given elements

    :param list Element: list of elements
    :param dict nsmap: namespace map

    :return dict: schema


Get url for describefeaturetype request

    :return str: url


use ComplexDataInput with a reference to a document


A URL that can be used to open the page.

        The URL is formatted from :py:attr:`URL_TEMPLATE`, which is then
        appended to :py:attr:`base_url` unless the template results in an
        absolute URL.

        :return: URL that can be used to open the page.
        :rtype: str


Open the page.

        Navigates to :py:attr:`seed_url` and calls :py:func:`wait_for_page_to_load`.

        :return: The current page object.
        :rtype: :py:class:`Page`
        :raises: UsageError


Root element for the page region.

        Page regions should define a root element either by passing this on
        instantiation or by defining a :py:attr:`_root_locator` attribute. To
        reduce the chances of hitting :py:class:`~selenium.common.exceptions.StaleElementReferenceException`
        or similar you should use :py:attr:`_root_locator`, as this is looked up every
        time the :py:attr:`root` property is accessed.


Finds an element on the page.

        :param strategy: Location strategy to use. See :py:class:`~selenium.webdriver.common.by.By` or :py:attr:`~pypom.splinter_driver.ALLOWED_STRATEGIES`.
        :param locator: Location of target element.
        :type strategy: str
        :type locator: str
        :return: An element.
        :rytpe: :py:class:`~selenium.webdriver.remote.webelement.WebElement` or :py:class:`~splinter.driver.webdriver.WebDriverElement`


Finds elements on the page.

        :param strategy: Location strategy to use. See :py:class:`~selenium.webdriver.common.by.By` or :py:attr:`~pypom.splinter_driver.ALLOWED_STRATEGIES`.
        :param locator: Location of target elements.
        :type strategy: str
        :type locator: str
        :return: List of :py:class:`~selenium.webdriver.remote.webelement.WebElement` or :py:class:`~splinter.element_list.ElementList`
        :rtype: list


Checks whether an element is present.

        :param strategy: Location strategy to use. See :py:class:`~selenium.webdriver.common.by.By` or :py:attr:`~pypom.splinter_driver.ALLOWED_STRATEGIES`.
        :param locator: Location of target element.
        :type strategy: str
        :type locator: str
        :return: ``True`` if element is present, else ``False``.
        :rtype: bool


Checks whether an element is displayed.

        :param strategy: Location strategy to use. See :py:class:`~selenium.webdriver.common.by.By` or :py:attr:`~pypom.splinter_driver.ALLOWED_STRATEGIES`.
        :param locator: Location of target element.
        :type strategy: str
        :type locator: str
        :return: ``True`` if element is displayed, else ``False``.
        :rtype: bool


Register driver adapter used by page object


Get the list of TV genres.

        Args:
            language: (optional) ISO 639-1 code.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the cast and crew information for a specific movie id.

        Args:
            append_to_response: (optional) Comma separated, any movie method.

        Returns:
            A dict representation of the JSON returned from the API.


Get the plot keywords for a specific movie id.

        Returns:
            A dict representation of the JSON returned from the API.


Get the release dates and certification for a specific movie id.

        Args:
            append_to_response: (optional) Comma separated, any movie method.

        Returns:
            A dict representation of the JSON returned from the API.


Get the translations for a specific movie id.

        Args:
            append_to_response: (optional) Comma separated, any movie method.

        Returns:
            A dict representation of the JSON returned from the API.


Get the similar movies for a specific movie id.

        Args:
            page: (optional) Minimum value of 1.  Expected value is an integer.
            language: (optional) ISO 639-1 code.
            append_to_response: (optional) Comma separated, any movie method.

        Returns:
            A dict representation of the JSON returned from the API.


Get the reviews for a particular movie id.

        Args:
            page: (optional) Minimum value of 1.  Expected value is an integer.
            language: (optional) ISO 639-1 code.
            append_to_response: (optional) Comma separated, any movie method.

        Returns:
            A dict representation of the JSON returned from the API.


Get the list of upcoming movies. This list refreshes every day.
        The maximum number of items this list will include is 100.

        Args:
            page: (optional) Minimum value of 1.  Expected value is an integer.
            language: (optional) ISO 639-1 code.

        Returns:
            A dict representation of the JSON returned from the API.


Get the list of movies playing in theatres. This list refreshes
        every day. The maximum number of items this list will include is 100.

        Args:
            page: (optional) Minimum value of 1.  Expected value is an integer.
            language: (optional) ISO 639-1 code.

        Returns:
            A dict representation of the JSON returned from the API.


Get the list of popular movies on The Movie Database. This list
        refreshes every day.

        Args:
            page: (optional) Minimum value of 1.  Expected value is an integer.
            language: (optional) ISO 639-1 code.

        Returns:
            A dict representation of the JSON returned from the API.


Get the list of top rated movies. By default, this list will only
        include movies that have 10 or more votes. This list refreshes every
        day.

        Args:
            page: (optional) Minimum value of 1.  Expected value is an integer.
            language: (optional) ISO 639-1 code.

        Returns:
            A dict representation of the JSON returned from the API.


This method lets users get the status of whether or not the movie has
        been rated or added to their favourite or watch lists. A valid session
        id is required.

        Args:
            session_id: see Authentication.

        Returns:
            A dict representation of the JSON returned from the API.


This method lets users rate a movie. A valid session id or guest
        session id is required.

        Args:
            session_id: see Authentication.
            guest_session_id: see Authentication.
            value: Rating value.

        Returns:
            A dict representation of the JSON returned from the API.


Get the movie credits for a specific person id.

        Args:
            language: (optional) ISO 639-1 code.
            append_to_response: (optional) Comma separated, any person method.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the detailed information about a particular credit record. This is
        currently only supported with the new credit model found in TV. These
        ids can be found from any TV credit response as well as the tv_credits
        and combined_credits methods for people.

        The episodes object returns a list of episodes and are generally going
        to be guest stars. The season array will return a list of season
        numbers.  Season credits are credits that were marked with the
        "add to every season" option in the editing interface and are
        assumed to be "season regulars".

        Args:
            language: (optional) ISO 639-1 code.

        Returns:
            A dict respresentation of the JSON returned from the API.


Discover TV shows by different types of data like average rating,
        number of votes, genres, the network they aired on and air dates.

        Args:
            page: (optional) Minimum 1, maximum 1000.
            language: (optional) ISO 639-1 code.
            sort_by: (optional) Available options are 'vote_average.desc',
                     'vote_average.asc', 'first_air_date.desc',
                     'first_air_date.asc', 'popularity.desc', 'popularity.asc'
            first_air_year: (optional) Filter the results release dates to
                            matches that include this value. Expected value
                            is a year.
            vote_count.gte or vote_count_gte: (optional) Only include TV shows
                            that are equal to,
                            or have vote count higher than this value. Expected
                            value is an integer.
            vote_average.gte or vote_average_gte: (optional) Only include TV
                              shows that are equal
                              to, or have a higher average rating than this
                              value.  Expected value is a float.
            with_genres: (optional) Only include TV shows with the specified
                         genres. Expected value is an integer (the id of a
                         genre).  Multiple valued can be specified. Comma
                         separated indicates an 'AND' query, while a
                         pipe (|) separated value indicates an 'OR'.
            with_networks: (optional) Filter TV shows to include a specific
                           network. Expected value is an integer (the id of a
                           network).  They can be comma separated to indicate an
                           'AND' query.
            first_air_date.gte or first_air_date_gte: (optional) The minimum
                                release to include.
                                Expected format is 'YYYY-MM-DD'.
            first_air_date.lte or first_air_date_lte: (optional) The maximum
                                release to include.
                                Expected format is 'YYYY-MM-DD'.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the system wide configuration info.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the list of supported certifications for movies.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the basic information for an account.

        Call this method first, before calling other Account methods.

        Returns:
            A dict respresentation of the JSON returned from the API.


Generate a session id for user based authentication.

        A session id is required in order to use any of the write methods.

        Args:
            request_token: The token you generated for the user to approve.
                           The token needs to be approved before being
                           used here.

        Returns:
            A dict respresentation of the JSON returned from the API.


Generate a guest session id.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get a list of rated moview for a specific guest session id.

        Args:
            page: (optional) Minimum 1, maximum 1000.
            sort_by: (optional) 'created_at.asc' | 'created_at.desc'
            language: (optional) ISO 639-1 code.

        Returns:
            A dict respresentation of the JSON returned from the API.


Delete movies from a list that the user created.

        A valid session id is required.

        Args:
            media_id: A movie id.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the similar TV series for a specific TV series id.

        Args:
            page: (optional) Minimum value of 1.  Expected value is an integer.
            language: (optional) ISO 639-1 code.
            append_to_response: (optional) Comma separated, any TV method.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the list of TV shows that are currently on the air. This query
        looks for any TV show that has an episode with an air date in the
        next 7 days.

        Args:
            page: (optional) Minimum 1, maximum 1000.
            language: (optional) ISO 639 code.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the primary information about a TV season by its season number.

        Args:
            language: (optional) ISO 639 code.
            append_to_response: (optional) Comma separated, any TV series
                                method.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the external ids that we have stored for a TV season by season
        number.

        Args:
            language: (optional) ISO 639 code.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the primary information about a TV episode by combination of a
        season and episode number.

        Args:
            language: (optional) ISO 639 code.
            append_to_response: (optional) Comma separated, any TV series
                                method.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the TV episode credits by combination of season and episode number.

        Returns:
            A dict respresentation of the JSON returned from the API.


Get the external ids for a TV episode by combination of a season and
        episode number.

        Args:
            language: (optional) ISO 639 code.

        Returns:
            A dict respresentation of the JSON returned from the API.


Search for movies by title.

        Args:
            query: CGI escpaed string.
            page: (optional) Minimum value of 1. Expected value is an integer.
            language: (optional) ISO 639-1 code.
            include_adult: (optional) Toggle the inclusion of adult titles.
                           Expected value is True or False.
            year: (optional) Filter the results release dates to matches that
                  include this value.
            primary_release_year: (optional) Filter the results so that only
                                  the primary release dates have this value.
            search_type: (optional) By default, the search type is 'phrase'.
                         This is almost guaranteed the option you will want.
                         It's a great all purpose search type and by far the
                         most tuned for every day querying. For those wanting
                         more of an "autocomplete" type search, set this
                         option to 'ngram'.

        Returns:
            A dict respresentation of the JSON returned from the API.


Search for people by name.

        Args:
            query: CGI escpaed string.
            page: (optional) Minimum value of 1. Expected value is an integer.
            include_adult: (optional) Toggle the inclusion of adult titles.
                           Expected value is True or False.
            search_type: (optional) By default, the search type is 'phrase'.
                         This is almost guaranteed the option you will want.
                         It's a great all purpose search type and by far the
                         most tuned for every day querying. For those wanting
                         more of an "autocomplete" type search, set this
                         option to 'ngram'.

        Returns:
            A dict respresentation of the JSON returned from the API.


Search for companies by name.

        Args:
            query: CGI escpaed string.
            page: (optional) Minimum value of 1. Expected value is an integer.

        Returns:
            A dict respresentation of the JSON returned from the API.


Search the movie, tv show and person collections with a single query.

        Args:
            query: CGI escpaed string.
            page: (optional) Minimum value of 1. Expected value is an integer.
            language: (optional) ISO 639-1 code.
            include_adult: (optional) Toggle the inclusion of adult titles.
                           Expected value is True or False.

        Returns:
            A dict respresentation of the JSON returned from the API.


r'-?\d+


r'\}


r'<<\S+\r?\n


Initialize the parse table at install time


Add another row of data from a test suite


Return an instance of SuiteFile, ResourceFile, SuiteFolder

    Exactly which is returned depends on whether it's a file or
    folder, and if a file, the contents of the file. If there is a
    testcase table, this will return an instance of SuiteFile,
    otherwise it will return an instance of ResourceFile.


The general idea is to do a quick parse, creating a list of
        tables. Each table is nothing more than a list of rows, with
        each row being a list of cells. Additional parsing such as
        combining rows into statements is done on demand. This first
        pass is solely to read in the plain text and organize it by table.


Return 'suite' or 'resource' or None

        This will return 'suite' if a testcase table is found;
        It will return 'resource' if at least one robot table
        is found. If no tables are found it will return None


Generator which returns all keywords in the suite


Regurgitate the tables and rows


Generator which returns all of the statements in all of the variables tables


The idea is, we recognize when we have a new testcase by
        checking the first cell. If it's not empty and not a comment,
        we have a new test case.


Parse command line arguments, and run rflint


Report a rule violation


Returns a list of rules of a given class

        Rules are treated as singletons - we only instantiate each
        rule once.


Import the given rule file


Handle the parsing of command line arguments.


Creates a customized Draft4ExtendedValidator.

        :param spec_resolver: resolver for the spec
        :type resolver: :class:`jsonschema.RefResolver`


While yaml supports integer keys, these are not valid in
        json, and will break jsonschema. This method coerces all keys
        to strings.


Open a file, read it and return its contents.


Takes a list of reference sentences for a single segment
    and returns an object that encapsulates everything that BLEU
    needs to know about them.


Takes a reference sentences for a single segment
    and returns an object that encapsulates everything that BLEU
    needs to know about them.  Also provides a set cause bleualign wants it


Creates the sentence alignment of two texts.

    Texts can consist of several blocks. Block boundaries cannot be crossed by sentence
    alignment links.

    Each block consists of a list that contains the lengths (in characters) of the sentences
    in this block.

    @param source_blocks: The list of blocks in the source text.
    @param target_blocks: The list of blocks in the target text.
    @param params: the sentence alignment parameters.

    @returns: A list of sentence alignment lists


Creates a path to model files
    model_path - string


Strips illegal characters from a string.  Used to sanitize input essays.
    Removes all non-punctuation, digit, or letter characters.
    Returns sanitized string.
    string - string


Uses aspell to spell correct an input string.
    Requires aspell to be installed and added to the path.
    Returns the spell corrected string if aspell is found, original string if not.
    string - string


Makes a list unique


Generates a count of the number of times each unique item appears in a list


Given an input string, part of speech tags the string, then generates a list of
    ngrams that appear in the string.
    Used to define grammatically correct part of speech tag sequences.
    Returns a list of part of speech tag sequences.


Generates predictions on a novel data array using a fit classifier
    clf is a classifier that has already been fit
    arr is a data array identical in dimension to the array clf was trained on
    Returns the array of predictions.


Calculates the average value of a list of numbers
    Returns a float


Calculates kappa correlation between rater_a and rater_b.
    Kappa measures how well 2 quantities vary together.
    rater_a is a list of rater a scores
    rater_b is a list of rater b scores
    min_rating is an optional argument describing the minimum rating possible on the data set
    max_rating is an optional argument describing the maximum rating possible on the data set
    Returns a float corresponding to the kappa correlation


Initializes dictionaries from an essay set object
        Dictionaries must be initialized prior to using this to extract features
        e_set is an input essay set
        returns a confirmation of initialization


Gets a set of gramatically correct part of speech sequences from an input file called essaycorpus.txt
        Returns the set and caches the file


Generates length based features from an essay set
        Generally an internal function called by gen_feats
        Returns an array of length features
        e_set - EssaySet object


Generates bag of words features from an input essay set and trained FeatureExtractor
        Generally called by gen_feats
        Returns an array of features
        e_set - EssaySet object


Generates bag of words, length, and prompt features from an essay set object
        returns an array of features
        e_set - EssaySet object