Pivot a dataframe by group of variables

    ---

    ### Parameters

    *mandatory :*
    * `variable` (*str*): name of the column used to create the groups.
    * `value` (*str*): name of the column containing the value to fill the pivoted df.
    * `new_columns` (*list of str*): names of the new columns.
    * `groups` (*dict*): names of the groups with their corresponding variables.
      **Warning**: the list of variables must have the same order as `new_columns`

    *optional :*
    * `id_cols` (*list of str*) : names of other columns to keep, default `None`.

    ---

    ### Example

    **Input**

    | type |  variable  | montant |
    |:----:|:----------:|:-------:|
    |   A  |    var1    |    5    |
    |   A  | var1_evol  |   0.3   |
    |   A  |    var2    |    6    |
    |   A  | var2_evol  |   0.2   |

    ```cson
    pivot_by_group :
      id_cols: ['type']
      variable: 'variable'
      value: 'montant'
      new_columns: ['value', 'variation']
      groups:
        'Group 1' : ['var1', 'var1_evol']
        'Group 2' : ['var2', 'var2_evol']
    ```

    **Ouput**

    | type |  variable  |  value  | variation |
    |:----:|:----------:|:-------:|:---------:|
    |   A  |   Group 1  |    5    |    0.3    |
    |   A  |   Group 2  |    6    |    0.2    |


Aggregate values by groups.

    ---

    ### Parameters

    *mandatory :*
    - `group_cols` (*list*): list of columns used to group data
    - `aggregations` (*dict*): dictionnary of values columns to group as keys and aggregation
      function to use as values (See the [list of aggregation functions](
      https://pandas.pydata.org/pandas-docs/stable/user_guide/groupby.html#aggregation))

    ---

    ### Example

    **Input**

    | ENTITY | YEAR | VALUE_1 | VALUE_2 |
    |:------:|:----:|:-------:|:-------:|
    |    A   | 2017 |    10   |    3    |
    |    A   | 2017 |    20   |    1    |
    |    A   | 2018 |    10   |    5    |
    |    A   | 2018 |    30   |    4    |
    |    B   | 2017 |    60   |    4    |
    |    B   | 2017 |    40   |    3    |
    |    B   | 2018 |    50   |    7    |
    |    B   | 2018 |    60   |    6    |

    ```cson
    groupby:
      group_cols: ['ENTITY', 'YEAR']
      aggregations:
        'VALUE_1': 'sum',
        'VALUE_2': 'mean'
    ```

    **Output**

    | ENTITY | YEAR | VALUE_1 | VALUE_2 |
    |:------:|:----:|:-------:|:-------:|
    |    A   | 2017 |    30   |   2.0   |
    |    A   | 2018 |    40   |   4.5   |
    |    B   | 2017 |   100   |   3.5   |
    |    B   | 2018 |   110   |   6.5   |


DEPRECATED - please use `compute_cumsum` instead


Decorator to catch an exception and don't raise it.
    Logs information if a decorator failed.

    Note:
        We don't want possible exceptions during logging to be raised.
        This is used to decorate any function that gets executed
        before or after the execution of the decorated function.


Replaces data values and column names according to the locale

    ---

    ### Parameters

    - `values` (optional: dict):
        - key: term to be replaced
        - value:
            - key: the locale e.g. 'en' or 'fr'
            - value: term's translation
    - `columns` (optional: dict):
        - key: columns name to be replaced
        - value:
            - key: the locale e.g. 'en' or 'fr'
            - value: column name's translation
    - `locale` (optional: str): the locale you want to use.
      By default the client locale is used.

    ---

    ### Example

    **Input**

    | label | value |
    |:----------------:|:-----:|
    | France | 100 |
    | Europe wo France | 500 |

    ```cson
    rename:
      values:
        'Europe wo France':
          'en': 'Europe excl. France'
          'fr': 'Europe excl. France'
      columns:
        'value':
          'en': 'revenue'
          'fr': 'revenue'
    ```

    **Output**

    | label | revenue |
    |:-------------------:|:-------:|
    | France | 100 |
    | Europe excl. France | 500 |


Aggregates data to reproduce "All" category for requester

    ---

    ### Parameters

    *mandatory :*
    - `id_cols` (*list*): the columns id to group
    - `cols_for_combination` (*dict*): colums corresponding to
       the filters as key and their default value as value

    *optional :*
    - `agg_func` (*str*, *list* or *dict*): the function(s) to use for aggregating the data.
       Accepted combinations are:
       - string function name
       - list of functions and/or function names, e.g. [np.sum, 'mean']
       - dict of axis labels -> functions, function names or list of such.


Get the value of a function's parameter based on its signature
    and the call's args and kwargs.

    Example:
        >>> def foo(a, b, c=3, d=4):
        ...    pass
        ...
        >>> # what would be the value of "c" when calling foo(1, b=2, c=33) ?
        >>> get_param_value_from_func_call('c', foo, [1], {'b': 2, 'c': 33})
        33


Creates aggregates following a given hierarchy

    ---

    ### Parameters

    *mandatory :*
    - `levels` (*list of str*): name of the columns composing the hierarchy (from the top to the bottom level).
    - `groupby_vars` (*list of str*): name of the columns with value to aggregate.
    - `extra_groupby_cols` (*list of str*) optional: other columns used to group in each level.

    *optional :*
    - `var_name` (*str*) : name of the result variable column. By default, `“type”`.
    - `value_name` (*str*): name of the result value column. By default, `“value”`.
    - `agg_func` (*str*): name of the aggregation operation. By default, `“sum”`.
    - `drop_levels` (*list of str*): the names of the levels that you may want to discard from the output.

    ---

    ### Example

    **Input**

    |    Region |     City |  Population |
    |:---------:|:--------:|:-----------:|
    |       Idf |     Panam|         200 |
    |       Idf |   Antony |          50 |
    |      Nord |    Lille |          20 |

    ```cson
    roll_up:
      levels: ["Region", "City"]
      groupby_vars: "Population"
    ```

    **Output**

    |    Region |     City |  Population |    value |   type |
    |:---------:|:--------:|:-----------:|:--------:|:------:|
    |       Idf |     Panam|         200 |    Panam |   City |
    |       Idf |   Antony |          50 |   Antony |   City |
    |      Nord |    Lille |          20 |    Lille |   City |
    |       Idf |      Nan |         250 |      Idf | Region |
    |      Nord |      Nan |          20 |     Nord | Region |


Keep the row of the data corresponding to the minimal value in a column

    ---

    ### Parameters

    *mandatory :*
    - `column` (str): name of the column containing the value you want to keep the minimum

    *optional :*
    - `groups` (*str or list(str)*): name of the column(s) used for 'groupby' logic
    (the function will return the argmax by group)
    ---

    ### Example

    **Input**

    | variable |   wave  |  year    | value |
    |:--------:|:-------:|:--------:|:-----:|
    |   toto   |  wave 1 |  2014    |  300  |
    |   toto   |  wave 1 |  2015    |  250  |
    |   toto   |  wave 1 |  2016    |  450  |

    ```cson
    argmin:
      column: 'year'
    ]
    ```

    **Output**

    | variable |   wave  |  year    | value |
    |:--------:|:-------:|:--------:|:-----:|
    |   toto   |  wave 1 |  2015    |  250  |


Can fill NaN values from a column with a given value or a column

    ---

    ### Parameters

    - `column` (*str*): name of column you want to fill
    - `value`: NaN will be replaced by this value
    - `column_value`: NaN will be replaced by value from this column

    *NOTE*: You must set either the 'value' parameter or the 'column_value' parameter

    ---

    ### Example

    **Input**

    | variable |   wave  |  year    | my_value |
    |:--------:|:-------:|:--------:|:--------:|
    |   toto   |  wave 1 |  2014    |  300     |
    |   toto   |  wave 1 |  2015    |          |
    |   toto   |  wave 1 |  2016    |  450     |

    ```cson
    fillna:
      column: 'my_value'
      value: 0
    ```

    **Output**

    | variable |   wave  |  year    | my_value |
    |:--------:|:-------:|:--------:|:--------:|
    |   toto   |  wave 1 |  2014    |  300     |
    |   toto   |  wave 1 |  2015    |    0     |
    |   toto   |  wave 1 |  2016    |  450     |


add a human readable offset to `dateobj` and return corresponding date.

    rely on `pandas.Timedelta` and add the following extra shortcuts:
    - "w", "week" and "weeks" for a week (i.e. 7days)
    - "month', "months" for a month (i.e. no day computation, just increment the month)
    - "y", "year', "years" for a year (i.e. no day computation, just increment the year)


return `dateobj` + `nb_months`

    If landing date doesn't exist (e.g. february, 30th), return the last
    day of the landing month.

    >>> add_months(date(2018, 1, 1), 1)
    datetime.date(2018, 1, 1)
    >>> add_months(date(2018, 1, 1), -1)
    datetime.date(2017, 12, 1)
    >>> add_months(date(2018, 1, 1), 25)
    datetime.date(2020, 2, 1)
    >>> add_months(date(2018, 1, 1), -25)
    datetime.date(2015, 12, 1)
    >>> add_months(date(2018, 1, 31), 1)
    datetime.date(2018, 2, 28)


return `dateobj` + `nb_years`

    If landing date doesn't exist (e.g. february, 30th), return the last
    day of the landing month.

    >>> add_years(date(2018, 1, 1), 1)
    datetime.date(2019, 1, 1)
    >>> add_years(date(2018, 1, 1), -1)
    datetime.date(2017, 1, 1)
    >>> add_years(date(2020, 2, 29), 1)
    datetime.date(2021, 2, 28)
    >>> add_years(date(2020, 2, 29), -1)
    datetime.date(2019, 2, 28)


parse `datestr` and return corresponding date object.

    `datestr` should be a string matching `date_fmt` and parseable by `strptime`
    but some offset can also be added using `(datestr) + OFFSET` or `(datestr) -
    OFFSET` syntax. When using this syntax, `OFFSET` should be understable by
    `pandas.Timedelta` (cf.
    http://pandas.pydata.org/pandas-docs/stable/timedeltas.html) and `w`, `week`
    `month` and `year` offset keywords are also accepted. `datestr` MUST be wrapped
    with parenthesis.

    Additionally, the following symbolic names are supported: `TODAY`,
    `YESTERDAY`, `TOMORROW`.

    Example usage:

    >>> parse_date('2018-01-01', '%Y-%m-%d') datetime.date(2018, 1, 1)
    parse_date('(2018-01-01) + 1day', '%Y-%m-%d') datetime.date(2018, 1, 2)
    parse_date('(2018-01-01) + 2weeks', '%Y-%m-%d') datetime.date(2018, 1, 15)

    Parameters: `datestr`: the date to parse, formatted as `date_fmt`
        `date_fmt`: expected date format

    Returns: The `date` object. If date could not be parsed, a ValueError will
        be raised.


Filter dataframe your data by date.

    This function will interpret `start`, `stop` and `atdate` and build
    the corresponding date range. The caller must specify either:

    - `atdate`: keep all rows matching this date exactly,
    - `start`: keep all rows matching this date onwards.
    - `stop`: keep all rows matching dates before this one.
    - `start` and `stop`: keep all rows between `start` and `stop`,

    Any other combination will raise an error. The lower bound of the date range
    will be included, the upper bound will be excluded.

    When specified, `start`, `stop` and `atdate` values are expected to match the
    `date_format` format or a known symbolic value (i.e. 'TODAY', 'YESTERDAY' or 'TOMORROW').

    Additionally, the offset syntax "(date) + offset" is also supported (Mind
    the parenthesis around the date string). In that case, the offset must be
    one of the syntax supported by `pandas.Timedelta` (see [pandas doc](
    http://pandas.pydata.org/pandas-docs/stable/timedeltas.html))

    ---

    ### Parameters

    *mandatory :*
    - `date_col` (*str*): the name of the dataframe's column to filter on

    *optional :*
    - `date_format` (*str*): expected date format in column `date_col` (see [available formats](
    https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior)
    - `start` (*str*): if specified, lower bound (included) of the date range
    - `stop` (*str*): if specified, upper bound (excluded) of the date range
    - `atdate` (*str*): if specified, the exact date we're filtering on


Add a column to the dataframe according to the groupby logic on group_cols

    ---

    ### Parameters

    *mandatory :*
    - `column` (*str*): name of the desired column you need percentage on

    *optional :*
    - `group_cols` (*list*): names of columns for the groupby logic
    - `new_column` (*str*): name of the output column. By default `column` will be overwritten.

    ---

    **Input**

    | gender |    sport   | number |
    |:------:|:----------:|:------:|
    |  male  |   bicycle  |   17   |
    | female | basketball |   17   |
    |  male  | basketball |    3   |
    | female |  football  |    7   |
    | female |   running  |   30   |
    |  male  |   running  |   20   |
    |  male  |  football  |   21   |
    | female |   bicycle  |   17   |

    ```cson
    percentage:
      new_column: 'number_percentage'
      column: 'number'
      group_cols: ['sport']
    ```

    **Output**

    | gender |    sport   | number | number_percentage |
    |:------:|:----------:|:------:|:-----------------:|
    |  male  |   bicycle  |   17   |        50.0       |
    | female | basketball |   17   |        85.0       |
    |  male  | basketball |    3   |        15.0       |
    | female |  football  |    7   |        25.0       |
    | female |   running  |   30   |        60.0       |
    |  male  |   running  |   20   |        40.0       |
    |  male  |  football  |   21   |        75.0       |
    | female |   bicycle  |   17   |        50.0       |


Get descriptor base path if string or return None.


Validate this Data Package.


Push Data Package to storage.

    All parameters should be used as keyword arguments.

    Args:
        descriptor (str): path to descriptor
        backend (str): backend name like `sql` or `bigquery`
        backend_options (dict): backend options mentioned in backend docs


Pull Data Package from storage.

    All parameters should be used as keyword arguments.

    Args:
        descriptor (str): path where to store descriptor
        name (str): name of the pulled datapackage
        backend (str): backend name like `sql` or `bigquery`
        backend_options (dict): backend options mentioned in backend docs


Convert resource's path and name to storage's table name.

    Args:
        path (str): resource path
        name (str): resource name

    Returns:
        str: table name


Restore schemas from being compatible with storage schemas.

    Foreign keys related operations.

    Args:
        list: resources from storage

    Returns:
        list: restored resources


It is possible for some of gdb's output to be read before it completely finished its response.
    In that case, a partial mi response was read, which cannot be parsed into structured data.
    We want to ALWAYS parse complete mi records. To do this, we store a buffer of gdb's
    output if the output did not end in a newline.

    Args:
        raw_output: Contents of the gdb mi output
        buf (str): Buffered gdb response from the past. This is incomplete and needs to be prepended to
        gdb's next output.

    Returns:
        (raw_output, buf)


Write to gdb process. Block while parsing responses from gdb for a maximum of timeout_sec.

        Args:
            mi_cmd_to_write (str or list): String to write to gdb. If list, it is joined by newlines.
            timeout_sec (float): Maximum number of seconds to wait for response before exiting. Must be >= 0.
            raise_error_on_timeout (bool): If read_response is True, raise error if no response is received
            read_response (bool): Block and read response. If there is a separate thread running,
            this can be false, and the reading thread read the output.
        Returns:
            List of parsed gdb responses if read_response is True, otherwise []
        Raises:
            NoGdbProcessError if there is no gdb subprocess running
            TypeError if mi_cmd_to_write is not valid


Get response from GDB, and block while doing so. If GDB does not have any response ready to be read
        by timeout_sec, an exception is raised.

        Args:
            timeout_sec (float): Maximum time to wait for reponse. Must be >= 0. Will return after
            raise_error_on_timeout (bool): Whether an exception should be raised if no response was found
            after timeout_sec

        Returns:
            List of parsed GDB responses, returned from gdbmiparser.parse_response, with the
            additional key 'stream' which is either 'stdout' or 'stderr'

        Raises:
            GdbTimeoutError if response is not received within timeout_sec
            ValueError if select returned unexpected file number
            NoGdbProcessError if there is no gdb subprocess running


Get responses on windows. Assume no support for select and use a while loop.


Get responses on unix-like system. Use select to wait for output.


Read count characters starting at self.index,
        and return those characters as a string


Parse gdb mi text and turn it into a dictionary.

    See https://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI-Stream-Records.html#GDB_002fMI-Stream-Records
    for details on types of gdb mi output.

    Args:
        gdb_mi_text (str): String output from gdb

    Returns:
        dict with the following keys:
        type (either 'notify', 'result', 'console', 'log', 'target', 'done'),
        message (str or None),
        payload (str, list, dict, or None)


Get notify message and payload dict


Optimize by SGD, AdaGrad, or AdaDelta.


Return updates in the training.


Get parameters to be optimized.


Compute first glimpse position using down-sampled image.


All codes that create parameters should be put into 'setup' function.


Build the computation graph here.


Process all data with given function.
        The scheme of function should be x,y -> x,y.


Make targets be one-hot vectors.


Print dataset statistics.


We train over mini-batches and evaluate periodically.


Sample outputs from LM.


Compute the alignment weights based on the previous state.


Pad sequences to given length in the left or right side.


RMSPROP optimization core.


Run the model with validation data and return costs.


This function will be called after each iteration.


Create inner loop variables.


Internal scan with dummy input variables.


Momentum SGD optimization core.


Skip N batches in the training.


Load parameters for the training.
        This method can load free parameters and resume the training progress.


Train the model and return costs.


Run one training iteration.


Run one valid iteration, return true if to continue training.


Get specified split of data.


Report usage of training parameters.


An alias of deepy.tensor.var.


Create vars given a dataset and set test values.
        Useful when dataset is already defined.


Create a shared theano scalar value.


Stack encoding layers, this must be done before stacking decoding layers.


Stack decoding layers.


Encode given input.


Register the layer so that it's param will be trained.
        But the output of the layer will not be stacked.


Monitoring the outputs of each layer.
        Useful for troubleshooting convergence problems.


Return all parameters.


Set up variables.


Save parameters to file.


Load parameters from file.


Print network statistics.


Register parameters.


Register updates that will only be executed in training phase.


Register monitors they should be tuple of name and Theano variable.


Get the L2 norm of multiple tensors.
    This function is taken from blocks.


dumps one element to file_obj, a file opened in write mode


Load parameters to the block.


Creates |oauth2| request elements.


We need to override this method to fix Facebooks naming deviation.


Google doesn't accept client ID and secret to be at the same time in
        request parameters and in the basic authorization header in the access
        token request.


Login handler, must accept both GET and POST to be able to use OpenID.


Replaces all values that are single-item iterables with the value of its
    index 0.

    :param dict dict_:
        Dictionary to normalize.

    :returns:
        Normalized dictionary.


Converts list of tuples to dictionary with duplicate keys converted to
    lists.

    :param list items:
        List of tuples.

    :returns:
        :class:`dict`


Parses response body from JSON, XML or query string.

    :param body:
        string

    :returns:
        :class:`dict`, :class:`list` if input is JSON or query string,
        :class:`xml.etree.ElementTree.Element` if XML.


Returns a provider class.

    :param class_name: :class:`string` or
    :class:`authomatic.providers.BaseProvider` subclass.


Creates the value for ``Set-Cookie`` HTTP header.

        :param bool delete:
            If ``True`` the cookie value will be ``deleted`` and the
            Expires value will be ``Thu, 01-Jan-1970 00:00:01 GMT``.


Creates signature for the session.


Converts the value to a signed string with timestamp.

        :param value:
            Object to be serialized.

        :returns:
            Serialized value.


``True`` if credentials are valid, ``False`` if expired.


Returns ``True`` if credentials expire sooner than specified.

        :param int seconds:
            Number of seconds.

        :returns:
            ``True`` if credentials expire sooner than specified,
            else ``False``.


Converts the credentials to a percent encoded string to be stored for
        later use.

        :returns:
            :class:`string`


Return true if string is binary data.


The whole response content.


Creates |oauth1| request elements.


Decorator for Flask view functions.


Launches the OpenID authentication procedure.


Logs a message with pre-formatted prefix.

        :param int level:
            Logging level as specified in the
            `login module <http://docs.python.org/2/library/logging.html>`_ of
            Python standard library.

        :param str msg:
            The actual message.


Splits given url to url base and params converted to list of tuples.


Deletes this worker's subscription.


Workers all share the same subscription so that tasks are
        distributed across all workers.


Enqueues a function for the task queue to execute.


Standalone PSQ worker.

    The queue argument must be the full importable path to a psq.Queue
    instance.

    Example usage:

        psqworker config.q

        psqworker --path /opt/app queues.fast


Gets the result of the task.

        Arguments:
            timeout: Maximum seconds to wait for a result before raising a
                TimeoutError. If set to None, this will wait forever. If the
                queue doesn't store results and timeout is None, this call will
                never return.


This function is the decorator which is used to wrap a Sanic route with.
    In the simplest case, simply use the default parameters to allow all
    origins in what is the most permissive configuration. If this method
    modifies state or performs authentication which may be brute-forced, you
    should add some degree of protection, such as Cross Site Forgery
    Request protection.

    :param origins:
        The origin, or list of origins to allow requests from.
        The origin(s) may be regular expressions, case-sensitive strings,
        or else an asterisk

        Default : '*'
    :type origins: list, string or regex

    :param methods:
        The method or list of methods which the allowed origins are allowed to
        access for non-simple requests.

        Default : [GET, HEAD, POST, OPTIONS, PUT, PATCH, DELETE]
    :type methods: list or string

    :param expose_headers:
        The header or list which are safe to expose to the API of a CORS API
        specification.

        Default : None
    :type expose_headers: list or string

    :param allow_headers:
        The header or list of header field names which can be used when this
        resource is accessed by allowed origins. The header(s) may be regular
        expressions, case-sensitive strings, or else an asterisk.

        Default : '*', allow all headers
    :type allow_headers: list, string or regex

    :param supports_credentials:
        Allows users to make authenticated requests. If true, injects the
        `Access-Control-Allow-Credentials` header in responses. This allows
        cookies and credentials to be submitted across domains.

        :note: This option cannot be used in conjuction with a '*' origin

        Default : False
    :type supports_credentials: bool

    :param max_age:
        The maximum time for which this CORS request maybe cached. This value
        is set as the `Access-Control-Max-Age` header.

        Default : None
    :type max_age: timedelta, integer, string or None

    :param send_wildcard: If True, and the origins parameter is `*`, a wildcard
        `Access-Control-Allow-Origin` header is sent, rather than the
        request's `Origin` header.

        Default : False
    :type send_wildcard: bool

    :param vary_header:
        If True, the header Vary: Origin will be returned as per the W3
        implementation guidelines.

        Setting this header when the `Access-Control-Allow-Origin` is
        dynamically generated (e.g. when there is more than one allowed
        origin, and an Origin than '*' is returned) informs CDNs and other
        caches that the CORS headers are dynamic, and cannot be cached.

        If False, the Vary header will never be injected or altered.

        Default : True
    :type vary_header: bool

    :param automatic_options:
        Only applies to the `cross_origin` decorator. If True, Sanic-CORS will
        override Sanic's default OPTIONS handling to return CORS headers for
        OPTIONS requests.

        Default : True
    :type automatic_options: bool


Performs the actual evaluation of Sanic-CORS options and actually
    modifies the response object.

    This function is used both in the decorator and the after_request
    callback
    :param sanic.request.Request req:


Wraps scalars or string types as a list, or returns the iterable instance.


Python 3.4 does not have math.isclose, so we need to steal it and add it here.


Deprecator decorator.


Attempts to deserialize a bytestring into an audiosegment.

    :param bstr: The bytestring serialized via an audiosegment's serialize() method.
    :returns: An AudioSegment object deserialized from `bstr`.


Returns an AudioSegment created from the given numpy array.

    The numpy array must have shape = (num_samples, num_channels).

    :param nparr: The numpy array to create an AudioSegment from.
    :returns: An AudioSegment created from the given array.


Executes a Sox command in a platform-independent manner.

        `cmd` must be a format string that includes {inputfile} and {outputfile}.


Returns a copy of this AudioSegment, but whose silence has been removed.

        .. note:: This method requires that you have the program 'sox' installed.

        .. warning:: This method uses the program 'sox' to perform the task. While this is very fast for a single
                     function call, the IO may add up for large numbers of AudioSegment objects.

        :param duration_s: The number of seconds of "silence" that must be present in a row to
                           be stripped.
        :param threshold_percentage: Silence is defined as any samples whose absolute value is below
                                     `threshold_percentage * max(abs(samples in this segment))`.
        :param console_output: If True, will pipe all sox output to the console.
        :returns: A copy of this AudioSegment, but whose silence has been removed.


Transforms the indicated slice of the AudioSegment into the frequency domain and returns the bins
        and the values.

        If neither `start_s` or `start_sample` is specified, the first sample of the slice will be the first sample
        of the AudioSegment.

        If neither `duration_s` or `num_samples` is specified, the slice will be from the specified start
        to the end of the segment.

        .. code-block:: python

            # Example for plotting the FFT using this function
            import matplotlib.pyplot as plt
            import numpy as np

            seg = audiosegment.from_file("furelise.wav")
            # Just take the first 3 seconds
            hist_bins, hist_vals = seg[1:3000].fft()
            hist_vals_real_normed = np.abs(hist_vals) / len(hist_vals)
            plt.plot(hist_bins / 1000, hist_vals_real_normed)
            plt.xlabel("kHz")
            plt.ylabel("dB")
            plt.show()

        .. image:: images/fft.png

        :param start_s: The start time in seconds. If this is specified, you cannot specify `start_sample`.
        :param duration_s: The duration of the slice in seconds. If this is specified, you cannot specify `num_samples`.
        :param start_sample: The zero-based index of the first sample to include in the slice.
                             If this is specified, you cannot specify `start_s`.
        :param num_samples: The number of samples to include in the slice. If this is specified, you cannot
                            specify `duration_s`.
        :param zero_pad: If True and the combination of start and duration result in running off the end of
                         the AudioSegment, the end is zero padded to prevent this.
        :returns: np.ndarray of frequencies in Hz, np.ndarray of amount of each frequency
        :raises: ValueError If `start_s` and `start_sample` are both specified and/or if both `duration_s` and
                            `num_samples` are specified.


Yields self's data in chunks of frame_duration_ms.

        This function adapted from pywebrtc's example [https://github.com/wiseman/py-webrtcvad/blob/master/example.py].

        :param frame_duration_ms: The length of each frame in ms.
        :param zero_pad: Whether or not to zero pad the end of the AudioSegment object to get all
                         the audio data out as frames. If not, there may be a part at the end
                         of the Segment that is cut off (the part will be <= `frame_duration_ms` in length).
        :returns: A Frame object with properties 'bytes (the data)', 'timestamp (start time)', and 'duration'.


Normalize the values in the AudioSegment so that its `spl` property
        gives `db`.

        .. note:: This method is currently broken - it returns an AudioSegment whose
                  values are much smaller than reasonable, yet which yield an SPL value
                  that equals the given `db`. Such an AudioSegment will not be serializable
                  as a WAV file, which will also break any method that relies on SOX.
                  I may remove this method in the future, since the SPL of an AudioSegment is
                  pretty questionable to begin with.

        :param db: The decibels to normalize average to.
        :returns: A new AudioSegment object whose values are changed so that their
                  average is `db`.
        :raises: ValueError if there are no samples in this AudioSegment.


Reduces others into this one by concatenating all the others onto this one and
        returning the result. Does not modify self, instead, makes a copy and returns that.

        :param others: The other AudioSegment objects to append to this one.
        :returns: The concatenated result.


Get the ID corresponding to the offset which occurs first after the given onset_front_id.
    By `first` I mean the front which contains the offset which is closest to the latest point
    in the onset front. By `after`, I mean that the offset must contain only offsets which
    occur after the latest onset in the onset front.

    If there is no appropriate offset front, the id returned is -1.


Gets an onset_front and an offset_front such that they both occupy at least some of the same
    frequency channels, then returns the portion of each that overlaps with the other.


Returns an updated segmentation mask such that the input `segmentation_mask` has been updated by segmenting between
    `onset_front_id` and `offset_front_id`, as found in `onset_fronts` and `offset_fronts`, respectively.

    This function also returns the onset_fronts and offset_fronts matrices, updated so that any fronts that are of
    less than 3 channels wide are removed.

    This function also returns a boolean value indicating whether the onset channel went to completion.

    Specifically, segments by doing the following:

    - Going across frequencies in the onset_front,
    - add the segment mask ID (the onset front ID) to all samples between the onset_front and the offset_front,
      if the offset_front is in that frequency.

    Possible scenarios:

    Fronts line up completely:

    ::

        |   |       S S S
        |   |  =>   S S S
        |   |       S S S
        |   |       S S S

    Onset front starts before offset front:

    ::

        |           |
        |   |       S S S
        |   |  =>   S S S
        |   |       S S S

    Onset front ends after offset front:

    ::

        |   |       S S S
        |   |  =>   S S S
        |   |       S S S
        |           |

    Onset front starts before and ends after offset front:

    ::

        |           |
        |   |  =>   S S S
        |   |       S S S
        |           |

    The above three options in reverse:

    ::

            |       |S S|           |
        |S S|       |S S|       |S S|
        |S S|       |S S|       |S S|
        |S S|           |           |

    There is one last scenario:

    ::

        |   |
        \   /
         \ /
         / \
        |   |

    Where the offset and onset fronts cross one another. If this happens, we simply
    reverse the indices and accept:

    ::

        |sss|
        \sss/
         \s/
         /s\
        |sss|

    The other option would be to destroy the offset front from the crossover point on, and
    then search for a new offset front for the rest of the onset front.


Removes all points in the fronts that overlap with the segmentation mask.


Removes all fronts from `fronts` which are strictly smaller than
    `size` consecutive frequencies in length.


For each onset front, for each frequency in that front, break the onset front if the signals
    between this frequency's onset and the next frequency's onset are not similar enough.

    Specifically:
    If we have the following two frequency channels, and the two O's are part of the same onset front,

    ::

        [ . O . . . . . . . . . . ]
        [ . . . . O . . . . . . . ]

    We compare the signals x and y:

    ::

        [ . x x x x . . . . . . . ]
        [ . y y y y . . . . . . . ]

    And if they are not sufficiently similar (via a DSP correlation algorithm), we break the onset
    front between these two channels.

    Once this is done, remove any onset fronts that are less than 3 channels wide.


Returns a list of segmentation masks each of the same dimension as the input one,
    but where they each have exactly one segment in them and all other samples in them
    are zeroed.

    Only bothers to return segments that are larger in total area than `threshold * mask.size`.


Worker for the ASA algorithm's multiprocessing step.


Does a lowpass filter over the given data.

    :param data: The data (numpy array) to be filtered.
    :param cutoff: The high cutoff in Hz.
    :param fs: The sample rate in Hz of the data.
    :param order: The order of the filter. The higher the order, the tighter the roll-off.
    :returns: Filtered data (numpy array).


Launch a Process, return his pid


Update the list of the running process and return the list


Give an IP, maybe a date, get the ASN.
            This is the fastest command.

            :param ip: IP address to search for
            :param announce_date: Date of the announcement

            :rtype: String, ASN.


Get the full history of an IP. It takes time.

            :param ip: IP address to search for
            :param days_limit: Max amount of days to query. (None means no limit)

            :rtype: list. For each day in the database: day, asn, block


Get the full history of an IP, aggregate the result instead of
            returning one line per day.

            :param ip: IP address to search for
            :param days_limit: Max amount of days to query. (None means no limit)

            :rtype: list. For each change: FirstDay, LastDay, ASN, Block


Inconditianilly download the URL in a temporary directory.
        When finished, the file is moved in the real directory.
        Like this an other process will not attempt to extract an inclomplete file.


Verify that the file has not already been downloaded.


Separates the outcome feature from the data and creates the onehot vector for each row.


Used to check whether the two edge lists have the same edges
  when elements are neither hashable nor sortable.


Given a list of audit files, rank them using the `measurer` and
  return the features that never deviate more than `similarity_bound`
  across repairs.


Loads a confusion matrix in a two-level dictionary format.

  For example, the confusion matrix of a 75%-accurate model
  that predicted 15 values (and mis-classified 5) may look like:
  {"A": {"A":10, "B": 5}, "B": {"B":5}}

  Note that raw boolean values are translated into strings, such that
  a value that was the boolean True will be returned as the string "True".


Separates the outcome feature from the data.


Renders a Page object as a Twitter Bootstrap styled pagination bar.
    Compatible with Bootstrap 3.x and 4.x only.

    Example::

        {% bootstrap_paginate page_obj range=10 %}

    Named Parameters::

        range - The size of the pagination bar (ie, if set to 10 then, at most,
                10 page numbers will display at any given time) Defaults to
                None, which shows all pages.

        size - Accepts "small", and "large". Defaults to
                    None which is the standard size.

        show_prev_next - Accepts "true" or "false". Determines whether or not
                        to show the previous and next page links. Defaults to
                        "true"

        show_first_last - Accepts "true" or "false". Determines whether or not
                          to show the first and last page links. Defaults to
                          "false"

        previous_label - The text to display for the previous page link.
                         Defaults to "←"

        next_label - The text to display for the next page link. Defaults to
                     "→"

        first_label - The text to display for the first page link. Defaults to
                      "«"

        last_label - The text to display for the last page link. Defaults to
                     "»"

        url_view_name - The named URL to use. Defaults to None. If None, then the
                        default template simply appends the url parameter as a
                        relative URL link, eg: 1

        url_param_name - The name of the parameter to use in the URL. If
                         url_view_name is set to None, this string is used as the
                         parameter name in the relative URL path. If a URL
                         name is specified, this string is used as the
                         parameter name passed into the reverse() method for
                         the URL.

        url_extra_args - This is used only in conjunction with url_view_name.
                         When referencing a URL, additional arguments may be
                         passed in as a list.

        url_extra_kwargs - This is used only in conjunction with url_view_name.
                           When referencing a URL, additional named arguments
                           may be passed in as a dictionary.

        url_get_params - The other get parameters to pass, only the page
                         number will be overwritten. Use this to preserve
                         filters.

        url_anchor - The anchor to use in URLs. Defaults to None.

        extra_pagination_classes - A space separated list of CSS class names
                                   that will be added to the top level

                                   HTML element. In particular, this can be
                                   utilized in Bootstrap 4 installatinos  to
                                   add the appropriate alignment classes from
                                   Flexbox utilites, eg:  justify-content-center


Checks for alternative index-url in pip.conf


For each package and target check if it is a regression.

    This is the case if the main repo contains a package version which is
    higher then in any of the other repos or if any of the other repos does not
    contain that package at all.

    :return: a dict indexed by package names containing
      dicts indexed by targets containing a boolean flag


Remove trailing junk from the version number.

    >>> strip_version_suffix('')
    ''
    >>> strip_version_suffix('None')
    'None'
    >>> strip_version_suffix('1.2.3-4trusty-20140131-1359-+0000')
    '1.2.3-4'
    >>> strip_version_suffix('1.2.3-foo')
    '1.2.3'


For each package check if the version in one repo is equal for all targets.

    The version could be different in different repos though.

    :return: a dict indexed by package names containing a boolean flag


Get the number of packages per target and repository.

    :return: a dict indexed by targets containing
      a list of integer values (one for each repo)


Get the Jenkins job urls for each target.

    The placeholder {pkg} needs to be replaced with the ROS package name.

    :return: a dict indexed by targets containing a string


Configure all Jenkins CI jobs.


Resolve all streams on the network.

    This function returns all currently available streams from any outlet on
    the network. The network is usually the subnet specified at the local
    router, but may also include a group of machines visible to each other via
    multicast packets (given that the network supports it), or list of
    hostnames. These details may optionally be customized by the experimenter
    in a configuration file (see Network Connectivity in the LSL wiki).

    Keyword arguments:
    wait_time -- The waiting time for the operation, in seconds, to search for
                 streams. Warning: If this is too short (<0.5s) only a subset
                 (or none) of the outlets that are present on the network may
                 be returned. (default 1.0)

    Returns a list of StreamInfo objects (with empty desc field), any of which
    can subsequently be used to open an inlet. The full description can be
    retrieved from the inlet.


Resolve all streams with a specific value for a given property.

    If the goal is to resolve a specific stream, this method is preferred over
    resolving all streams and then selecting the desired one.

    Keyword arguments:
    prop -- The StreamInfo property that should have a specific value (e.g.,
            "name", "type", "source_id", or "desc/manufaturer").
    value -- The string value that the property should have (e.g., "EEG" as
             the type property).
    minimum -- Return at least this many streams. (default 1)
    timeout -- Optionally a timeout of the operation, in seconds. If the
               timeout expires, less than the desired number of streams
               (possibly none) will be returned. (default FOREVER)

    Returns a list of matching StreamInfo objects (with empty desc field), any
    of which can subsequently be used to open an inlet.

    Example: results = resolve_Stream_byprop("type","EEG")


Resolve all streams that match a given predicate.

    Advanced query that allows to impose more conditions on the retrieved
    streams; the given string is an XPath 1.0 predicate for the <description>
    node (omitting the surrounding []'s), see also
    http://en.wikipedia.org/w/index.php?title=XPath_1.0&oldid=474981951.

    Keyword arguments:
    predicate -- The predicate string, e.g. "name='BioSemi'" or
                "type='EEG' and starts-with(name,'BioSemi') and
                 count(description/desc/channels/channel)=32"
    minimum -- Return at least this many streams. (default 1)
    timeout -- Optionally a timeout of the operation, in seconds. If the
               timeout expires, less than the desired number of streams
               (possibly none) will be returned. (default FOREVER)

    Returns a list of matching StreamInfo objects (with empty desc field), any
    of which can subsequently be used to open an inlet.


Error handler function. Translates an error code into an exception.


Get a child with a specified name.


Get the previous sibling in the children list of the parent node.

        If a name is provided, the previous sibling with the given name is
        returned.


Set the element's name. Returns False if the node is empty.


Set the element's value. Returns False if the node is empty.


Append a copy of the specified element as a child.


Remove a given child element, specified by name or as element.


Obtain the set of currently present streams on the network.

        Returns a list of matching StreamInfo objects (with empty desc
        field), any of which can subsequently be used to open an inlet.


See all token associated with a given token.
    PAIR lilas


Shows autocomplete results for a given token.


Compute fuzzy extensions of word.
    FUZZY lilas


Compute fuzzy extensions of word that exist in index.
    FUZZYINDEX lilas


Try to extract the bigger group of interlinked tokens.

    Should generally be used at last in the collectors chain.


Display this help message.


Print some useful infos from Redis DB.


Print raw content of a DB key.
        DBKEY g|u09tyzfe


Compute a geohash from latitude and longitude.
        GEOHASH 48.1234 2.9876


Get index details for a document by its id.
        INDEX 772210180J


Return document linked to word with higher score.
        BESTSCORE lilas


Print the distance score between two strings. Use | as separator.
        STRDISTANCE rue des lilas|porte des lilas


Just sends the request using its send method and returns its response.


Concurrently converts a list of Requests to Responses.

    :param requests: a collection of Request objects.
    :param stream: If False, the content will not be downloaded immediately.
    :param size: Specifies the number of workers to run at a time. If 1, no parallel processing.
    :param exception_handler: Callback function, called when exception occured. Params: Request, Exception


Removes PEM-encoding from a public key, private key or certificate. If the
    private key is encrypted, the password will be used to decrypt it.

    :param data:
        A byte string of the PEM-encoded data

    :param password:
        A byte string of the encryption password, or None

    :return:
        A 3-element tuple in the format: (key_type, algorithm, der_bytes). The
        key_type will be a unicode string of "public key", "private key" or
        "certificate". The algorithm will be a unicode string of "rsa", "dsa"
        or "ec".


Decrypts encrypted ASN.1 data

    :param encryption_algorithm_info:
        An instance of asn1crypto.pkcs5.Pkcs5EncryptionAlgorithm

    :param encrypted_content:
        A byte string of the encrypted content

    :param password:
        A byte string of the encrypted content's password

    :return:
        A byte string of the decrypted plaintext


Creates an EVP_CIPHER pointer object and determines the buffer size
    necessary for the parameter specified.

    :param evp_cipher_ctx:
        An EVP_CIPHER_CTX pointer

    :param cipher:
        A unicode string of "aes128", "aes192", "aes256", "des",
        "tripledes_2key", "tripledes_3key", "rc2", "rc4"

    :param key:
        The key byte string

    :param data:
        The plaintext or ciphertext as a byte string

    :param padding:
        If padding is to be used

    :return:
        A 2-element tuple with the first element being an EVP_CIPHER pointer
        and the second being an integer that is the required buffer size


Takes a CryptoAPI RSA private key blob and converts it into the ASN.1
    structures for the public and private keys

    :param bit_size:
        The integer bit size of the key

    :param blob_struct:
        An instance of the advapi32.RSAPUBKEY struct

    :param blob:
        A byte string of the binary data after the header

    :return:
        A 2-element tuple of (asn1crypto.keys.PublicKeyInfo,
        asn1crypto.keys.PrivateKeyInfo)


Takes a CryptoAPI DSS private key blob and converts it into the ASN.1
    structures for the public and private keys

    :param bit_size:
        The integer bit size of the key

    :param public_blob:
        A byte string of the binary data after the public key header

    :param private_blob:
        A byte string of the binary data after the private key header

    :return:
        A 2-element tuple of (asn1crypto.keys.PublicKeyInfo,
        asn1crypto.keys.PrivateKeyInfo)


Generates a DSA signature

    :param private_key:
        The PrivateKey to generate the signature with

    :param data:
        A byte string of the data the signature is for

    :param hash_algorithm:
        A unicode string of "md5", "sha1", "sha256", "sha384" or "sha512"

    :raises:
        ValueError - when any of the parameters contain an invalid value
        TypeError - when any of the parameters are of the wrong type
        OSError - when an error is returned by the OS crypto library

    :return:
        A byte string of the signature


Generates an ECDSA signature

    :param private_key:
        The PrivateKey to generate the signature with

    :param data:
        A byte string of the data the signature is for

    :param hash_algorithm:
        A unicode string of "md5", "sha1", "sha256", "sha384" or "sha512"

    :raises:
        ValueError - when any of the parameters contain an invalid value
        TypeError - when any of the parameters are of the wrong type
        OSError - when an error is returned by the OS crypto library

    :return:
        A byte string of the signature


Generates an RSA, DSA or ECDSA signature via CryptoAPI

    :param private_key:
        The PrivateKey to generate the signature with

    :param data:
        A byte string of the data the signature is for

    :param hash_algorithm:
        A unicode string of "md5", "sha1", "sha256", "sha384", "sha512" or "raw"

    :param rsa_pss_padding:
        If PSS padding should be used for RSA keys

    :raises:
        ValueError - when any of the parameters contain an invalid value
        TypeError - when any of the parameters are of the wrong type
        OSError - when an error is returned by the OS crypto library

    :return:
        A byte string of the signature


Generates an RSA, DSA or ECDSA signature via CNG

    :param private_key:
        The PrivateKey to generate the signature with

    :param data:
        A byte string of the data the signature is for

    :param hash_algorithm:
        A unicode string of "md5", "sha1", "sha256", "sha384", "sha512" or "raw"

    :param rsa_pss_padding:
        If PSS padding should be used for RSA keys

    :raises:
        ValueError - when any of the parameters contain an invalid value
        TypeError - when any of the parameters are of the wrong type
        OSError - when an error is returned by the OS crypto library

    :return:
        A byte string of the signature


Encrypts a value using an RSA public key via CNG

    :param certificate_or_public_key:
        A Certificate or PublicKey instance to encrypt with

    :param data:
        A byte string of the data to encrypt

    :param rsa_oaep_padding:
        If OAEP padding should be used instead of PKCS#1 v1.5

    :raises:
        ValueError - when any of the parameters contain an invalid value
        TypeError - when any of the parameters are of the wrong type
        OSError - when an error is returned by the OS crypto library

    :return:
        A byte string of the ciphertext


Encrypts a value using an RSA private key via CryptoAPI

    :param private_key:
        A PrivateKey instance to decrypt with

    :param ciphertext:
        A byte string of the data to decrypt

    :param rsa_oaep_padding:
        If OAEP padding should be used instead of PKCS#1 v1.5

    :raises:
        ValueError - when any of the parameters contain an invalid value
        TypeError - when any of the parameters are of the wrong type
        OSError - when an error is returned by the OS crypto library

    :return:
        A byte string of the plaintext


Blocks until the socket is ready to be read from, or the timeout is hit

        :param timeout:
            A float - the period of time to wait for data to be read. None for
            no time limit.

        :return:
            A boolean - if data is ready to be read. Will only be False if
            timeout is not None.


Reads exactly the specified number of bytes from the socket

        :param num_bytes:
            An integer - the exact number of bytes to read

        :return:
            A byte string of the data that was read


Reads data from the socket and writes it to the memory bio
        used by libssl to decrypt the data. Returns the unencrypted
        data for the purpose of debugging handshakes.

        :return:
            A byte string of ciphertext from the socket. Used for
            debugging the handshake only.


Takes ciphertext from the memory bio and writes it to the
        socket.

        :return:
            A byte string of ciphertext going to the socket. Used
            for debugging the handshake only.


Encrypts plaintext via CNG

    :param cipher:
        A unicode string of "aes", "des", "tripledes_2key", "tripledes_3key",
        "rc2", "rc4"

    :param key:
        The encryption key - a byte string 5-16 bytes long

    :param data:
        The plaintext - a byte string

    :param iv:
        The initialization vector - a byte string - unused for RC4

    :param padding:
        Boolean, if padding should be used - unused for RC4

    :raises:
        ValueError - when any of the parameters contain an invalid value
        TypeError - when any of the parameters are of the wrong type
        OSError - when an error is returned by the OS crypto library

    :return:
        A byte string of the ciphertext


Checks if an error occured, and if so throws an OSError containing the
    last OpenSSL error message

    :param result:
        An integer result code - 1 or greater indicates success

    :param exception_class:
        The exception class to use for the exception if an error occurred

    :raises:
        OSError - when an OpenSSL error occurs


Return the certificate and a hash of it

    :param cert_pointer:
        A SecCertificateRef

    :return:
        A 2-element tuple:
         - [0]: A byte string of the SHA1 hash of the cert
         - [1]: A byte string of the DER-encoded contents of the cert


Extracts the last OS error message into a python unicode string

    :return:
        A unicode string error message


Converts a CFDictionary object into a python dictionary

        :param dictionary:
            The CFDictionary to convert

        :return:
            A python dict


Extracts the function signature and description of a Python function

    :param docstring:
        A unicode string of the docstring for the function

    :param def_lineno:
        An integer line number that function was defined on

    :param code_lines:
        A list of unicode string lines from the source file the function was
        defined in

    :param prefix:
        A prefix to prepend to all output lines

    :return:
        A 2-element tuple:

         - [0] A unicode string of the function signature with a docstring of
               parameter info
         - [1] A markdown snippet of the function description


Walks through a CommonMark AST to find section headers that delineate
    content that should be updated by this script

    :param md_ast:
        The AST of the markdown document

    :param sections:
        A dict to store the start and end lines of a section. The key will be
        a two-element tuple of the section type ("class", "function",
        "method" or "attribute") and identifier. The values are a two-element
        tuple of the start and end line number in the markdown document of the
        section.

    :param last:
        A dict containing information about the last section header seen.
        Includes the keys "type_name", "identifier", "start_line".

    :param last_class:
        A unicode string of the name of the last class found - used when
        processing methods and attributes.

    :param total_lines:
        An integer of the total number of lines in the markdown document -
        used to work around a bug in the API of the Python port of CommonMark


A callback used to walk the Python AST looking for classes, functions,
    methods and attributes. Generates chunks of markdown markup to replace
    the existing content.

    :param node:
        An _ast module node object

    :param code_lines:
        A list of unicode strings - the source lines of the Python file

    :param sections:
        A dict of markdown document sections that need to be updated. The key
        will be a two-element tuple of the section type ("class", "function",
        "method" or "attribute") and identifier. The values are a two-element
        tuple of the start and end line number in the markdown document of the
        section.

    :param md_chunks:
        A dict with keys from the sections param and the values being a unicode
        string containing a chunk of markdown markup.


Tries to find a CA certs bundle in common locations

    :raises:
        OSError - when no valid CA certs bundle was found on the filesystem

    :return:
        The full filesystem path to a CA certs bundle file


Extracts trusted CA certs from the system CA cert bundle

    :param cert_callback:
        A callback that is called once for each certificate in the trust store.
        It should accept two parameters: an asn1crypto.x509.Certificate object,
        and a reason. The reason will be None if the certificate is being
        exported, otherwise it will be a unicode string of the reason it won't.

    :param callback_only_on_failure:
        A boolean - if the callback should only be called when a certificate is
        not exported.

    :return:
        A list of 3-element tuples:
         - 0: a byte string of a DER-encoded certificate
         - 1: a set of unicode strings that are OIDs of purposes to trust the
              certificate for
         - 2: a set of unicode strings that are OIDs of purposes to reject the
              certificate for


Parse the TLS handshake from the client to the server to extract information
    including the cipher suite selected, if compression is enabled, the
    session id and if a new or reused session ticket exists.

    :param server_handshake_bytes:
        A byte string of the handshake data received from the server

    :param client_handshake_bytes:
        A byte string of the handshake data sent to the server

    :return:
        A dict with the following keys:
         - "protocol": unicode string
         - "cipher_suite": unicode string
         - "compression": boolean
         - "session_id": "new", "reused" or None
         - "session_ticket: "new", "reused" or None


Creates a generator returning tuples of information about each record
    in a byte string of data from a TLS client or server. Stops as soon as it
    find a ChangeCipherSpec message since all data from then on is encrypted.

    :param data:
        A byte string of TLS records

    :return:
        A generator that yields 3-element tuples:
        [0] Byte string of record type
        [1] Byte string of protocol version
        [2] Byte string of record data


Creates a generator returning tuples of information about each message in
    a byte string of data from a TLS handshake record

    :param data:
        A byte string of a TLS handshake record data

    :return:
        A generator that yields 2-element tuples:
        [0] Byte string of message type
        [1] Byte string of message data


Creates a generator returning tuples of information about each extension
    from a byte string of extension data contained in a ServerHello ores
    ClientHello message

    :param data:
        A byte string of a extension data from a TLS ServerHello or ClientHello
        message

    :return:
        A generator that yields 2-element tuples:
        [0] Byte string of extension type
        [1] Byte string of extension data


Raises a TLSVerificationError due to a hostname mismatch

    :param certificate:
        An asn1crypto.x509.Certificate object

    :raises:
        TLSVerificationError


Raises a TLSVerificationError due to certificate being expired, or not yet
    being valid

    :param certificate:
        An asn1crypto.x509.Certificate object

    :raises:
        TLSVerificationError


Looks at the server handshake bytes to try and detect a different protocol

    :param server_handshake_bytes:
        A byte string of the handshake data received from the server

    :return:
        None, or a unicode string of "ftp", "http", "imap", "pop3", "smtp"


Reads everything available from the socket - used for debugging when there
    is a protocol error

    :param socket:
        The socket to read from

    :return:
        A byte string of the remaining data


Takes a set of unicode string OIDs and converts vendor-specific OIDs into
    generics OIDs from RFCs.

     - 1.2.840.113635.100.1.3 (apple_ssl) -> 1.3.6.1.5.5.7.3.1 (server_auth)
     - 1.2.840.113635.100.1.3 (apple_ssl) -> 1.3.6.1.5.5.7.3.2 (client_auth)
     - 1.2.840.113635.100.1.8 (apple_smime) -> 1.3.6.1.5.5.7.3.4 (email_protection)
     - 1.2.840.113635.100.1.9 (apple_eap) -> 1.3.6.1.5.5.7.3.13 (eap_over_ppp)
     - 1.2.840.113635.100.1.9 (apple_eap) -> 1.3.6.1.5.5.7.3.14 (eap_over_lan)
     - 1.2.840.113635.100.1.11 (apple_ipsec) -> 1.3.6.1.5.5.7.3.5 (ipsec_end_system)
     - 1.2.840.113635.100.1.11 (apple_ipsec) -> 1.3.6.1.5.5.7.3.6 (ipsec_tunnel)
     - 1.2.840.113635.100.1.11 (apple_ipsec) -> 1.3.6.1.5.5.7.3.7 (ipsec_user)
     - 1.2.840.113635.100.1.11 (apple_ipsec) -> 1.3.6.1.5.5.7.3.17 (ipsec_ike)
     - 1.2.840.113635.100.1.16 (apple_code_signing) -> 1.3.6.1.5.5.7.3.3 (code_signing)
     - 1.2.840.113635.100.1.20 (apple_time_stamping) -> 1.3.6.1.5.5.7.3.8 (time_stamping)
     - 1.3.6.1.4.1.311.10.3.2 (microsoft_time_stamp_signing) -> 1.3.6.1.5.5.7.3.8 (time_stamping)

    :param oids:
        A set of unicode strings

    :return:
        The original set of OIDs with any mapped OIDs added


Checks to see if a cache file needs to be refreshed

    :param ca_path:
        A unicode string of the path to the cache file

    :param cache_length:
        An integer representing the number of hours the cache is valid for

    :return:
        A boolean - True if the cache needs to be updated, False if the file
        is up-to-date


Gets value of bits between selected range from memory

    :param start: bit address of start of bit of bits
    :param end: bit address of first bit behind bits
    :return: instance of BitsVal (derived from SimBits type) which contains
        copy of selected bits


Cast HArray signal or value to signal or value of type Bits


Hdl convertible in operator, check if any of items
    in "iterable" equals "sigOrVal"


Logical shift left


Returns no of bits required to store x-1
    for example x=8 returns 3


c-like case of switch statement


c-like default of switch statement


Register signals from interfaces for Interface or Unit instances


This method is called for every value change of any signal.


Serialize HWProcess instance

        :param scope: name scope to prevent name collisions


Walk all interfaces on unit and instantiate agent for every interface.

    :return: all monitor/driver functions which should be added to simulation
         as processes


If interface has associated clk return it otherwise
        try to find clk on parent recursively


same like itertools.groupby

    :note: This function does not needs initial sorting like itertools.groupby

    :attention: Order of pairs is not deterministic.


Flatten nested lists, tuples, generators and maps

    :param level: maximum depth of flattening


If signal is not driving anything remove it


Try merge procB into procA

    :raise IncompatibleStructure: if merge is not possible
    :attention: procA is now result if merge has succeed
    :return: procA which is now result of merge


on writeReqRecieved in monitor mode


Convert unit to RTL using specified serializer

    :param unitOrCls: unit instance or class, which should be converted
    :param name: name override of top unit (if is None name is derived
        form class name)
    :param serializer: serializer which should be used for to RTL conversion
    :param targetPlatform: metainformatins about target platform, distributed
        on every unit under _targetPlatform attribute
        before Unit._impl() is called
    :param saveTo: directory where files should be stored
        If None RTL is returned as string.
    :raturn: if saveTo returns RTL string else returns list of file names
        which were created


Create new signal in this context

        :param clk: clk signal, if specified signal is synthesized
            as SyncSignal
        :param syncRst: synchronous reset signal


Get maximum _instId from all assigments in statement


get max statement id,
    used for sorting of processes in architecture


write data to interface


Note that this interface will be master

        :return: self


load declaratoins from _declr method
        This function is called first for parent and then for children


generate _sig for each interface which has no subinterface
        if already has _sig return it instead

        :param context: instance of RtlNetlist where signals should be created
        :param prefix: name prefix for created signals
        :param typeTransform: optional function (type) returns modified type
            for signal


Get name in HDL


Load all operands and process them by self._evalFn


Cast signed-unsigned, to int or bool


Reinterpret signal of type Bits to signal of type HStruct


Group transaction parts splited on words to words

        :param transaction: TransTmpl instance which parts
            should be grupped into words
        :return: generator of tuples (wordIndex, list of transaction parts
            in this word)


Pretty print interface


Convert transaction template into FrameTmpls

        :param transaction: transaction template used which are FrameTmpls
            created from
        :param wordWidth: width of data signal in target interface
            where frames will be used
        :param maxFrameLen: maximum length of frame in bits,
            if exceeded another frame will be created
        :param maxPaddingWords: maximum of continual padding words in frame,
            if exceed frame is split and words are cut of
        :attention: if maxPaddingWords<inf trimPaddingWordsOnEnd
            or trimPaddingWordsOnStart has to be True
            to decide where padding should be trimmed
        :param trimPaddingWordsOnStart: trim padding from start of frame
            at word granularity
        :param trimPaddingWordsOnEnd: trim padding from end of frame
            at word granularity


Walk enumerated words in this frame

        :attention: not all indexes has to be present, only words
            with items will be generated when not showPadding
        :param showPadding: padding TransParts are also present
        :return: generator of tuples (wordIndex, list of TransParts
            in this word)


Pack data into list of BitsVal of specified dataWidth

        :param data: dict of values for struct fields {fieldName: value}

        :return: list of BitsVal which are representing values of words


Clean informations about enclosure for outputs and sensitivity
        of this statement


Discover sensitivity for list of signals


get RtlNetlist context from signals


Update signal IO after reuce atempt

        :param self_reduced: if True this object was reduced
        :param io_changed: if True IO of this object may changed
            and has to be updated
        :param result_statements: list of statements which are result
            of reduce operation on this statement


After merging statements update IO, sensitivity and context

        :attention: rank is not updated


Merge statements in list to remove duplicated if-then-else trees

        :return: tuple (list of merged statements, rank decrease due merging)
        :note: rank decrease is sum of ranks of reduced statements
        :attention: statement list has to me mergable


Simplify statements in the list


After parrent statement become event dependent
        propagate event dependency flag to child statements


Append statements to this container under conditions specified
        by condSet


Disconnect this statement from signals and delete it from RtlNetlist context

        :attention: signal endpoints/drivers will be altered
            that means they can not be used for iteration


Create register in this unit

        :param defVal: default value of this register,
            if this value is specified reset of this component is used
            (unit has to have single interface of class Rst or Rst_n)
        :param clk: optional clok signal specification
        :param rst: optional reset signal specification
        :note: rst/rst_n resolution is done from signal type,
            if it is negated type it is rst_n
        :note: if clk or rst is not specifid default signal
            from parent unit will be used


Create signal in this unit


Walk all simple values in HStruct or HArray


Convert signum, no bit manipulation just data are represented
        differently

        :param signed: if True value will be signed,
            if False value will be unsigned,
            if None value will be vector without any sign specification


register sensitivity for process


Evaluate list of values as condition


Connect ports of simulation models by name


Create value updater for simulation

    :param nextVal: instance of Value which will be asssiggned to signal
    :param invalidate: flag which tells if value has been compromised
        and if it should be invaidated
    :return: function(value) -> tuple(valueHasChangedFlag, nextVal)


Create value updater for simulation for value of array type

    :param nextVal: instance of Value which will be asssiggned to signal
    :param indexes: tuple on indexes where value should be updated
        in target array

    :return: function(value) -> tuple(valueHasChangedFlag, nextVal)


set value of this param


Resolve ports of discovered memories


Construct value of this type.
        Delegated on value class for this type


Cast value or signal of this type to another type of same size.

        :param sigOrVal: instance of signal or value to cast
        :param toType: instance of HdlType to cast into


Concatenate all signals to one big signal, recursively

    :param masterDirEqTo: only signals with this direction are packed
    :param exclude: sequence of signals/interfaces to exclude


Return sig and val reduced by & operator or None
    if it is not possible to statically reduce expression


Return sig and val reduced by ^ operator or None
    if it is not possible to statically reduce expression


Get root of name space


Decide if this unit should be serialized or not eventually fix name
        to fit same already serialized unit

        :param obj: object to serialize
        :param serializedClasses: dict {unitCls : unitobj}
        :param serializedConfiguredUnits: (unitCls, paramsValues) : unitObj
            where paramsValues are named tuple name:value


Serialize HdlType instance


Srialize IfContainer instance


Get constant name for value
        name of constant is reused if same value was used before


Cut off statements which are driver of specified signal


Parse HArray type to this transaction template instance

        :return: address of it's end


Only for transactions derived from HArray

        :return: width of item in original array


Walk fields in instance of TransTmpl

        :param offset: optional offset for all children in this TransTmpl
        :param shouldEnterFn: function (transTmpl) which returns True
            when field should be split on it's children
        :param shouldEnterFn: function(transTmpl) which should return
            (shouldEnter, shouldUse) where shouldEnter is flag that means
            iterator should look inside of this actual object
            and shouldUse flag means that this field should be used
            (=generator should yield it)
        :return: generator of tuples ((startBitAddress, endBitAddress),
            TransTmpl instance)


Merge other statement to this statement


Cached indent getter function


Check if not redefining property on obj


Register interface object on interface level object


Register array of items on interface level object


Returns a first driver if signal has only one driver.


Recursively statistically evaluate result of this operator


Create operator with result signal

        :ivar resT: data type of result signal
        :ivar outputs: iterable of singnals which are outputs
            from this operator


Try connect src to interface of specified name on unit.
    Ignore if interface is not present or if it already has driver.


Propagate "clk" clock signal to all subcomponents


Propagate "clk" clock and negative reset "rst_n" signal
    to all subcomponents


Propagate reset "rst" signal
    to all subcomponents


Iterate over bits in vector

    :param sigOrVal: signal or value to iterate over
    :param bitsInOne: number of bits in one part
    :param skipPadding: if true padding is skipped in dense types


Always decide not to serialize obj

    :param priv: private data for this function first unit of this class
    :return: tuple (do serialize this object, next priv)


Decide to serialize only first obj of it's class

    :param priv: private data for this function
        (first object with class == obj.__class__)

    :return: tuple (do serialize this object, next priv)
        where priv is private data for this function
        (first object with class == obj.__class__)


Decide to serialize only objs with uniq parameters and class

    :param priv: private data for this function
        ({frozen_params: obj})

    :return: tuple (do serialize this object, next priv)


Delegate _make_association on items

        :note: doc in :func:`~hwt.synthesizer.interfaceLevel.propDeclCollector._make_association`


Create a simulation model for unit

    :param unit: interface level unit which you wont prepare for simulation
    :param targetPlatform: target platform for this synthes
    :param dumpModelIn: folder to where put sim model files
        (otherwise sim model will be constructed only in memory)


Reconnect model signals to unit to run simulation with simulation model
    but use original unit interfaces for communication

    :param synthesisedUnitOrIntf: interface where should be signals
        replaced from signals from modelCls
    :param modelCls: simulation model form where signals
        for synthesisedUnitOrIntf should be taken


Syntax sugar
    If outputFile is string try to open it as file

    :return: hdl simulator object


Process for injecting of this callback loop into simulator


Connect internal signal to port item,
        this connection is used by simulator and only output port items
        will be connected


connet signal from internal side of of this component to this port


return signal inside unit which has this port


Schedule process on actual time with specified priority


Add hdl process to execution queue

        :param trigger: instance of SimSignal
        :param proc: python generator function representing HDL process


Schedule combUpdateDoneEv event to let agents know that current
        delta step is ending and values from combinational logic are stable


Apply stashed values to signals


This functions resolves write conflicts for signal

        :param actionSet: set of actions made by process


Delta step for combinational processes


Read value from signal or interface


Write value to signal or interface.


Convert all ternary operators to IfContainers


Create a new version under this service.


Create a new VCL under this version.


Converts the column to a dictionary representation accepted
        by the Citrination server.

        :return: Dictionary with basic options, plus any column type specific
            options held under the "options" key
        :rtype: dict


Add a descriptor column.

        :param descriptor: A Descriptor instance (e.g., RealDescriptor, InorganicDescriptor, etc.)
        :param role: Specify a role (input, output, latentVariable, or ignore)
        :param group_by_key: Whether or not to group by this key during cross validation


Checks to see that the query will not exceed the max query depth

        :param returning_query: The PIF system or Dataset query to execute.
        :type returning_query: :class:`PifSystemReturningQuery` or :class: `DatasetReturningQuery`


Run each in a list of PIF queries against Citrination.

        :param multi_query: :class:`MultiQuery` object to execute.
        :return: :class:`PifMultiSearchResult` object with the results of the query.


Updates an existing data view from the search template and ml template given

        :param id: Identifier for the data view.  This returned from the create method.
        :param configuration: Information to construct the data view from (eg descriptors, datasets etc)
        :param name: Name of the data view
        :param description: Description for the data view


Gets basic information about a view

        :param data_view_id: Identifier of the data view
        :return: Metadata about the view as JSON


Creates an ml configuration from dataset_ids and extract_as_keys

        :param dataset_ids: Array of dataset identifiers to make search template from
        :return: An identifier used to request the status of the builder job (get_ml_configuration_status)


Utility function to turn the result object from the configuration builder endpoint into something that
        can be used directly as a configuration.

        :param result_blob: Nested dicts representing the possible descriptors
        :param dataset_ids: Array of dataset identifiers to make search template from
        :return: An object suitable to be used as a parameter to data view create


After invoking the create_ml_configuration async method, you can use this method to
        check on the status of the builder job.

        :param job_id: The identifier returned from create_ml_configuration
        :return: Job status


Get the t-SNE projection, including responses and tags.

        :param data_view_id: The ID of the data view to retrieve TSNE from
        :type data_view_id: int
        :return: The TSNE analysis
        :rtype: :class:`Tsne`


Submits an async prediction request.

        :param data_view_id: The id returned from create
        :param candidates: Array of candidates
        :param prediction_source: 'scalar' or 'scalar_from_distribution'
        :param use_prior: True to use prior prediction, otherwise False
        :return: Predict request Id (used to check status)


Returns a string indicating the status of the prediction job

        :param view_id: The data view id returned from data view create
        :param predict_request_id: The id returned from predict
        :return: Status data, also includes results if state is finished


Submits a new experimental design run.

        :param data_view_id: The ID number of the data view to which the
            run belongs, as a string
        :type data_view_id: str
        :param num_candidates: The number of candidates to return
        :type num_candidates: int
        :param target: An :class:``Target`` instance representing
            the design run optimization target
        :type target: :class:``Target``
        :param constraints: An array of design constraints (instances of
            objects which extend :class:``BaseConstraint``)
        :type constraints: list of :class:``BaseConstraint``
        :param sampler: The name of the sampler to use during the design run:
            either "Default" or "This view"
        :type sampler: str
        :return: A :class:`DesignRun` instance containing the UID of the
            new run


Retrieves a summary of information for a given data view
            - view id
            - name
            - description
            - columns

        :param data_view_id: The ID number of the data view to which the
            run belongs, as a string
        :type data_view_id: str


Given a filepath, loads the file as a dictionary from YAML

    :param path: The path to a YAML file


Extracts credentials from the yaml formatted credential filepath
    passed in. Uses the default profile if the CITRINATION_PROFILE env var
    is not set, otherwise looks for a profile with that name in the credentials file.

    :param filepath: The path of the credentials file


Given an API key, a site url and a credentials file path, runs through a prioritized list of credential sources to find credentials.

    Specifically, this method ranks credential priority as follows:
        1. Those passed in as the first two parameters to this method
        2. Those found in the environment as variables
        3. Those found in the credentials file at the profile specified
           by the profile environment variable
        4. Those found in the default stanza in the credentials file

    :param api_key: A Citrination API Key or None
    :param site: A Citrination site URL or None
    :param cred_file: The path to a credentials file


Returns the number of files matching a pattern in a dataset.

        :param dataset_id: The ID of the dataset to search for files.
        :type dataset_id: int
        :param glob: A pattern which will be matched against files in the dataset.
        :type glob: str
        :param is_dir: A boolean indicating whether or not the pattern should match against the beginning of paths in the dataset.
        :type is_dir: bool
        :return: The number of matching files
        :rtype: int


Retrieves a PIF from a given dataset.

        :param dataset_id: The id of the dataset to retrieve PIF from
        :type dataset_id: int
        :param uid: The uid of the PIF to retrieve
        :type uid: str
        :param dataset_version: The dataset version to look for the PIF in. If nothing is supplied, the latest dataset version will be searched
        :type dataset_version: int
        :return: A :class:`Pif` object
        :rtype: :class:`Pif`


Retrieves the set of columns from the combination of dataset ids given

        :param dataset_ids: The id of the dataset to retrieve columns from
        :type dataset_ids: list of int
        :return: A list of column names from the dataset ids given.
        :rtype: list of str


Generates a default search templates from the available columns in the dataset ids given.

        :param dataset_ids: The id of the dataset to retrieve files from
        :type dataset_ids: list of int
        :return: A search template based on the columns in the datasets given


Returns a new search template, but the new template has only the extract_as_keys given.

        :param extract_as_keys: List of extract as keys to keep
        :param search_template: The search template to prune
        :return: New search template with pruned columns


Make a copy of a dictionary with all keys converted to camel case. This is just calls to_camel_case on each of the keys in the dictionary and returns a new dictionary.

        :param obj: Dictionary to convert keys to camel case.
        :return: Dictionary with the input values and all keys in camel case


Runs the template against the validation endpoint, returns a message indicating status of the templte

        :param ml_template: Template to validate
        :return: OK or error message if validation failed


Compute the hamming distance between two hashes


Compute the average hash of the given image.


Set up the Vizio media player platform.


Retrieve latest state of the device.


Mute the volume.


Increasing volume of the device.


Decreasing volume of the device.


Restores the starting position.


Gets the piece at the given square.


Removes a piece from the given square if present.


Sets a piece at the given square. An existing piece is replaced.


Checks if the given move would move would leave the king in check or
        put it into check.


Checks if the king of the other side is attacked. Such a position is not
        valid and could only be reached by an illegal move.


Checks if the current position is a checkmate.


a game is ended if a position occurs for the fourth time
        on consecutive alternating moves.


Restores the previous position and returns the last move from the stack.


Gets an SFEN representation of the current position.


Parses a move in standard coordinate notation, makes the move and puts
        it on the the move stack.
        Raises `ValueError` if neither legal nor a null move.
        Returns the move.


Returns a Zobrist hash of the current position.


Gets the symbol `p`, `l`, `n`, etc.


Creates a piece instance from a piece symbol.
        Raises `ValueError` if the symbol is invalid.


Parses an USI string.
        Raises `ValueError` if the USI string is invalid.


Accept a string and parse it into many commits.
    Parse and yield each commit-dictionary.
    This function is a generator.


Accept a parsed single commit. Some of the named groups
    require further processing, so parse those groups.
    Return a dictionary representing the completely parsed
    commit.


Adds a organization-course link to the system


Course key object validation


Inactivates an activated organization as well as any active relationships


Activates an inactive organization-course relationship


Inactivates an active organization-course relationship


Retrieves the set of courses currently linked to the specified organization


Retrieves the organizations linked to the specified course


Organization dict-to-object serialization


Load's config then runs Django's execute_from_command_line


Adds argument for config to existing argparser


Find config file and set values


Dumps initial config in YAML


Documents values in markdown


converts string to type requested by `cast_as`


\
        loop through all the images and find the ones
        that have the best bytez to even make them a candidate


\
        checks to see if we were able to
        find open link_src on this page


\
        returns the bytes of the image file on disk


Create a video object from a video embed


adds any siblings that may have a decent score to this node


\
        returns a list of nodes we want to search
        on like paragraphs and tables


\
        remove any divs that looks like non-content,
        clusters of links, or paras with no gusto


\
        Fetch the article title and analyze it


if the article has meta canonical link set in the url


Close the network connection and perform any other required cleanup

            Note:
                Auto closed when using goose as a context manager or when garbage collected


Extract the most likely article content from the html page

            Args:
                url (str): URL to pull and parse
                raw_html (str): String representation of the HTML page
            Returns:
                Article: Representation of the article contents \
                including other parsed and extracted metadata


Returns a unicode object representing 's'. Treats bytestrings using the
    'encoding' codec.

    If strings_only is True, don't convert (some) non-string-like objects.


Returns a bytestring version of 's', encoded as specified in 'encoding'.

    If strings_only is True, don't convert (some) non-string-like objects.


Add URLs needed to handle image uploads.


Handle file uploads from WYSIWYG.


Render the Quill WYSIWYG.


Get the form for field.


Resize an image for metadata tags, and return an absolute URL to it.


Check if ``mdrun`` finished successfully.

    Analyses the output from ``mdrun`` in *logfile*. Right now we are
    simply looking for the line "Finished mdrun on node" in the last 1kb of
    the file. (The file must be seeakable.)

    :Arguments:
      *logfile* : filename
         Logfile produced by ``mdrun``.

    :Returns: ``True`` if all ok, ``False`` if not finished, and
              ``None`` if the *logfile* cannot be opened


Launch local smpd.


Find files from a continuation run


Run ``gromacs.grompp`` and return the total charge of the  system.

    :Arguments:
       The arguments are the ones one would pass to :func:`gromacs.grompp`.
    :Returns:
       The total charge as reported

    Some things to keep in mind:

    * The stdout output of grompp is only shown when an error occurs. For
      debugging, look at the log file or screen output and try running the
      normal :func:`gromacs.grompp` command and analyze the output if the
      debugging messages are not sufficient.

    * Check that ``qtot`` is correct. Because the function is based on pattern
      matching of the informative output of :program:`grompp` it can break when
      the output format changes. This version recognizes lines like ::

            '  System has non-zero total charge: -4.000001e+00'

      using the regular expression
      :regexp:`System has non-zero total charge: *(?P<qtot>[-+]?\d*\.\d+([eE][-+]\d+)?)`.


Create a processed topology.

    The processed (or portable) topology file does not contain any
    ``#include`` statements and hence can be easily copied around. It
    also makes it possible to re-grompp without having any special itp
    files available.

    :Arguments:
      *topol*
          topology file
      *struct*
          coordinat (structure) file

    :Keywords:
      *processed*
          name of the new topology file; if not set then it is named like
          *topol* but with ``pp_`` prepended
      *includes*
          path or list of paths of directories in which itp files are
          searched for
      *grompp_kwargs**
          other options for :program:`grompp` such as ``maxwarn=2`` can
          also be supplied

    :Returns: full path to the processed topology


Primitive text file stream editor.

    This function can be used to edit free-form text files such as the
    topology file. By default it does an **in-place edit** of
    *filename*. If *newname* is supplied then the edited
    file is written to *newname*.

    :Arguments:
       *filename*
           input text file
       *substitutions*
           substitution commands (see below for format)
       *newname*
           output filename; if ``None`` then *filename* is changed in
           place [``None``]

    *substitutions* is a list of triplets; the first two elements are regular
    expression strings, the last is the substitution value. It mimics
    ``sed`` search and replace. The rules for *substitutions*:

    .. productionlist::
       substitutions: "[" search_replace_tuple, ... "]"
       search_replace_tuple: "(" line_match_RE "," search_RE "," replacement ")"
       line_match_RE: regular expression that selects the line (uses match)
       search_RE: regular expression that is searched in the line
       replacement: replacement string for search_RE

    Running :func:`edit_txt` does pretty much what a simple ::

         sed /line_match_RE/s/search_RE/replacement/

    with repeated substitution commands does.

    Special replacement values:
    - ``None``: the rule is ignored
    - ``False``: the line is deleted (even if other rules match)

    .. note::

       * No sanity checks are performed and the substitutions must be supplied
         exactly as shown.
       * All substitutions are applied to a line; thus the order of the substitution
         commands may matter when one substitution generates a match for a subsequent rule.
       * If replacement is set to ``None`` then the whole expression is ignored and
         whatever is in the template is used. To unset values you must provided an
         empty string or similar.
       * Delete a matching line if replacement=``False``.


Delete all frames.


Returns resid in the Gromacs index by transforming with offset.


Combine individual groups into a single one and write output.

        :Keywords:
           name_all : string
              Name of the combined group, ``None`` generates a name.  [``None``]
           out_ndx : filename
              Name of the output file that will contain the individual groups
              and the combined group. If ``None`` then default from the class
              constructor is used. [``None``]
           operation : character
              Logical operation that is used to generate the combined group from
              the individual groups: "|" (OR) or "&" (AND); if set to ``False``
              then no combined group is created and only the individual groups
              are written. ["|"]
           defaultgroups : bool
              ``True``: append everything to the default groups produced by
              :program:`make_ndx` (or rather, the groups provided in the ndx file on
              initialization --- if this was ``None`` then these are truly default groups);
              ``False``: only use the generated groups

        :Returns:
           ``(combinedgroup_name, output_ndx)``, a tuple showing the
           actual group name and the name of the file; useful when all names are autogenerated.

        .. Warning:: The order of the atom numbers in the combined group is
                     *not* guaranteed to be the same as the selections on input because
                     ``make_ndx`` sorts them ascending. Thus you should be careful when
                     using these index files for calculations of angles and dihedrals.
                     Use :class:`gromacs.formats.NDX` in these cases.

        .. SeeAlso:: :meth:`IndexBuilder.write`.


Concatenate input index files.

        Generate a new index file that contains the default Gromacs index
        groups (if a structure file was defined) and all index groups from the
        input index files.

        :Arguments:
           out_ndx : filename
              Name of the output index file; if ``None`` then use the default
              provided to the constructore. [``None``].


Process ``make_ndx`` command and  return name and temp index file.


Process a range selection.

        ("S234", "A300", "CA")   --> selected all CA in this range
        ("S234", "A300")         --> selected all atoms in this range

        .. Note:: Ignores residue type, only cares about the resid (but still required)


Translate selection for a single res to make_ndx syntax.


Simple tests to flag problems with a ``make_ndx`` run.


Write compact xtc that is fitted to the tpr reference structure.

        See :func:`gromacs.cbook.trj_fitandcenter` for details and
        description of *kwargs* (including *input*, *input1*, *n* and
        *n1* for how to supply custom index groups). The most important ones are listed
        here but in most cases the defaults should work.

        :Keywords:
           *s*
             Input structure (typically the default tpr file but can be set to
             some other file with a different conformation for fitting)
           *n*
             Alternative index file.
           *o*
             Name of the output trajectory.
           *xy* : Boolean
             If ``True`` then only fit in xy-plane (useful for a membrane normal
             to z). The default is ``False``.
           *force*
             - ``True``: overwrite existing trajectories
             - ``False``: throw a IOError exception
             - ``None``: skip existing and log a warning [default]

        :Returns:
              dictionary with keys *tpr*, *xtc*, which are the names of the
              the new files


Write xtc that is fitted to the tpr reference structure.

        Runs :class:`gromacs.tools.trjconv` with appropriate arguments
        for fitting. The most important *kwargs* are listed
        here but in most cases the defaults should work.

        Note that the default settings do *not* include centering or
        periodic boundary treatment as this often does not work well
        with fitting. It is better to do this as a separate step (see
        :meth:`center_fit` or :func:`gromacs.cbook.trj_fitandcenter`)

        :Keywords:
           *s*
             Input structure (typically the default tpr file but can be set to
             some other file with a different conformation for fitting)
           *n*
             Alternative index file.
           *o*
             Name of the output trajectory. A default name is created.
             If e.g. *dt* = 100  is one of the *kwargs* then the default name includes
             "_dt100ps".
          *xy* : boolean
             If ``True`` then only do a rot+trans fit in the xy plane
             (good for membrane simulations); default is ``False``.
          *force*
            ``True``: overwrite existing trajectories
            ``False``: throw a IOError exception
            ``None``: skip existing and log a warning [default]
          *fitgroup*
            index group to fit on ["backbone"]

            .. Note:: If keyword *input* is supplied then it will override
                      *fitgroup*; *input* = ``[fitgroup, outgroup]``
          *kwargs*
             kwargs are passed to :func:`~gromacs.cbook.trj_xyfitted`

        :Returns:
              dictionary with keys *tpr*, *xtc*, which are the names of the
              the new files