transducers

System Information

Definition Index

• TRANSDUCERS

  Ergonomic, efficient data processing.

  • EXTERNAL CONDITION

    EMPTY-TRANSDUCTION

    Source

    A transduction was empty when it was expected not to be.

  • EXTERNAL CONDITION

    IMBALANCED-PLIST

    Source

    A given `plist' source had an uneven number of keys.

  • EXTERNAL STRUCTURE

    PLIST

    Source

    No documentation provided.
  • EXTERNAL FUNCTION

    ALLP

    • PRED

    Source

    Reducer: Yield non-NIL if all elements of the transduction satisfy PRED.
    Short-circuits with NIL if any element fails the test.

  • EXTERNAL FUNCTION

    ANYP

    • PRED

    Source

    Reducer: Yield non-NIL if any element in the transduction satisfies PRED.
    Short-circuits the transduction as soon as the condition is met.

  • EXTERNAL FUNCTION

    AVERAGE

    • &OPTIONAL
    • ACC
    • INPUT

    Source

    Reducer: Calculate the average value of all numeric elements in a transduction.

  • EXTERNAL FUNCTION

    BRANCH

    • PRED
    • TA
    • TB

    Source

    Transducer: If a PRED yields non-NIL on a value, proceed with transducer chain
    TA. Otherwise, follow chain TB. This produces a kind of diamond pattern of data
    flow within the transduction:
    
         /4a-5a-6a\
    1-2-3          7-8-9
         \4b-5b---/
    
    Assuming that TA here is some composition of three transducer steps, and TB is a
    composition of two. Naturally, if you have other steps beyond the fork (Step 7
    above), you should make sure that they can handle the return values of both
    sides!
    
    (transduce (comp (map #'1+)
                     (branch #'evenp
                             (map (comp #'write-to-string #'1+))
                             (map (const "Odd!")))
                     (map #'length))
               #'cons (range 1 6))
    => (1 4 1 4 1)
    

  • EXTERNAL FUNCTION

    COMP

    • FUNCTION
    • &REST
    • FUNCTIONS

    Source

    Function composition.
    
    (funcall (comp #'1+ #'length) "foo") == (1+ (length "foo"))

  • EXTERNAL FUNCTION

    CONCATENATE

    • REDUCER

    Source

    Transducer: Concatenate all the sublists in the transduction.

  • EXTERNAL FUNCTION

    CONS

    • &OPTIONAL
    • ACC
    • INPUT

    Source

    Reducer: Collect all results as a list.

  • EXTERNAL FUNCTION

    CONST

    • ITEM

    Source

    Return a function that ignores its argument and returns ITEM instead.

  • EXTERNAL FUNCTION

    COUNT

    • &OPTIONAL
    • ACC
    • INPUT

    Source

    Reducer: Count the number of elements that made it through the transduction.

  • EXTERNAL FUNCTION

    DEDUP

    • REDUCER

    Source

    Transducer: Remove adjacent duplicates from the transduction.

  • EXTERNAL FUNCTION

    DROP

    • N

    Source

    Transducer: Drop the first N elements of the transduction.

  • EXTERNAL FUNCTION

    DROP-WHILE

    • PRED

    Source

    Transducer: Drop elements from the front of the transduction that satisfy PRED.

  • EXTERNAL FUNCTION

    ENUMERATE

    • REDUCER

    Source

    Transducer: Index every value passed through the transduction into a cons pair.
    Starts at 0.

  • EXTERNAL FUNCTION

    FILTER

    • PRED

    Source

    Transducer: Only keep elements from the transduction that satisfy PRED.

  • EXTERNAL FUNCTION

    FILTER-MAP

    • F

    Source

    Transducer: Apply a function F to the elements of the transduction, but only
    keep results that are non-nil.
    
    (transduce (filter-map #'cl:first) #'cons '(() (2 3) () (5 6) () (8 9)))
    => (2 5 8)
    

  • EXTERNAL FUNCTION

    FIND

    • PRED

    Source

    Reducer: Find the first element in the transduction that satisfies a given PRED.
    Yields `nil' if no such element were found.

  • EXTERNAL FUNCTION

    FIRST

    • &OPTIONAL
    • ACC
    • INPUT

    Source

    Reducer: Yield the first value of the transduction. As soon as this first value
    is yielded, the entire transduction stops.
    
    # Conditions
    
    - `empty-transduction': when no values made it through the transduction.
    

  • EXTERNAL FUNCTION

    FLATTEN

    • REDUCER

    Source

    Transducer: Entirely flatten all lists in the transduction, regardless of
    nesting.

  • EXTERNAL FUNCTION

    FOLD

    • F
    • &OPTIONAL
    • SEED

    Source

    Reducer: The fundamental reducer. `fold' creates an ad-hoc reducer based on
    a given 2-argument function. An optional SEED value can also be given as the
    initial accumulator value, which also becomes the return value in case there
    were no input left in the transduction.
    
    Functions like `+' and `*' are automatically valid reducers, because they yield
    sane values even when given 0 or 1 arguments. Other functions like `max' cannot
    be used as-is as reducers since they can't be called without arguments. For
    functions like this, `fold' is appropriate.
    
    # Conditions
    
    - `empty-transduction': if no SEED is given and the transduction is empty.
    

  • EXTERNAL FUNCTION

    FOR-EACH

    • &REST
    • VARGS

    Source

    Reducer: Run through every item in a transduction for their side effects.
    Throws away all results and yields t.

  • EXTERNAL FUNCTION

    FROM-CSV

    • REDUCER

    Source

    Transducer: Interpret the data stream as CSV data.
    
    The first item found is assumed to be the header list, and it will be used to
    construct useable hashtables for all subsequent items.
    
    Note: This function makes no attempt to convert types from the
    original parsed strings. If you want numbers, you will need to
    further parse them yourself.
    
    This function is expected to be passed "bare" to `transduce', so there is no
    need for the caller to manually pass a REDUCER.

  • EXTERNAL FUNCTION

    GROUP-BY

    • F

    Source

    Transducer: Group the input stream into sublists via some function F. The cutoff
    criterion is whether the return value of F changes between two consecutive elements of the
    transduction.
    
    (transduce (group-by #'evenp) #'cons '(2 4 6 7 9 1 2 4 6 3))
    => ((2 4 6) (7 9 1) (2 4 6) (3))
    

  • EXTERNAL FUNCTION

    HASH-TABLE

    • &OPTIONAL
    • ACC
    • INPUT

    Source

    Reducer: Collect a stream of key-value cons pairs into a hash table.

  • EXTERNAL FUNCTION

    INJECT

    • F

    Source

    Transducer: For each value in the transduction that actually affects the final
    result (tested with `EQ'), inject an extra transduction step into the chain
    immediately after this point. Accumulates, such that each new injection appears
    before the previous one.

  • EXTERNAL FUNCTION

    INTERSPERSE

    • ELEM

    Source

    Transducer: Insert an ELEM between each value of the transduction.

  • EXTERNAL FUNCTION

    INTO-CSV

    • HEADERS

    Source

    Transducer: Given a sequence of HEADERS, rerender each item in the data stream
    into a CSV string. It's assumed that each item in the transduction is a hash
    table whose keys are strings that match the values found in HEADERS.
    
    # Conditions
    
    - `empty-argument': when an empty HEADERS sequence is given.
    

  • EXTERNAL FUNCTION

    INTS

    • START
    • &KEY
    • STEP

    Source

    Source: Yield all integers, beginning with START and advancing by an optional
    STEP value which can be positive or negative. If you only want a specific range
    within the transduction, then use `take-while' within your transducer chain.

  • EXTERNAL FUNCTION

    LAST

    • &OPTIONAL
    • ACC
    • INPUT

    Source

    Reducer: Yield the last value of the transduction.
    
    # Conditions
    
    - `empty-transduction': when no values made it through the transduction.
    

  • EXTERNAL FUNCTION

    LOG

    • LOGGER

    Source

    Transducer: Call some LOGGER function for each step of the transduction. The
    LOGGER must accept the running results and the current element as input. The
    original results of the transduction are passed through as-is.

  • EXTERNAL FUNCTION

    MAKE-REDUCED

    • &KEY
    • VAL

    Source

    No documentation provided.
  • EXTERNAL FUNCTION

    MAP

    • F

    Source

    Transducer: Apply a function F to all elements of the transduction.

  • EXTERNAL FUNCTION

    MAX

    • DEFAULT

    Source

    Deprecated: Use `fold' and pass it `cl:max' instead.

  • EXTERNAL FUNCTION

    MIN

    • DEFAULT

    Source

    Deprecated: Use `fold' and pass it `cl:min' instead.

  • EXTERNAL FUNCTION

    ONCE

    • ITEM

    Source

    Transducer: Inject some ITEM into the front of the transduction.

  • EXTERNAL FUNCTION

    PASS

    • REDUCER

    Source

    Transducer: Just pass along each value of the transduction. Same in intent with
    applying `map' to `identity', but this should be slightly more efficient. It is
    at least shorter to type.

  • EXTERNAL FUNCTION

    PLIST

    • PLIST

    Source

    Source: Yield key-value pairs from a Property List, usually known as a 'plist'.

  • EXTERNAL FUNCTION

    RANDOM

    • LIMIT

    Source

    Source: Yield an endless stream of random numbers.

  • EXTERNAL FUNCTION

    REDUCED-P

    • OBJECT

    Source

    No documentation provided.
  • EXTERNAL FUNCTION

    REDUCED-VAL

    • INSTANCE

    Source

    No documentation provided.
  • EXTERNAL FUNCTION

    (SETF REDUCED-VAL)

    • VALUE
    • INSTANCE

    Source

    No documentation provided.
  • EXTERNAL FUNCTION

    REPEAT

    • ITEM

    Source

    Source: Endlessly yield a given ITEM.

  • EXTERNAL FUNCTION

    SCAN

    • F
    • SEED

    Source

    Transducer: Build up successsive values from the results of previous
    applications of a given function F.
    
    (transduce (scan #'+ 0) #'cons '(1 2 3 4))
    => (0 1 3 6 10)

  • EXTERNAL FUNCTION

    SEGMENT

    • N

    Source

    Transducer: Partition the input into lists of N items. If the input stops, flush
    any accumulated state, which may be shorter than N.
    
    # Conditions
    
    - `non-positive-integer': when a non-positive integer N is given.
    

  • EXTERNAL FUNCTION

    SHUFFLE

    • VEC

    Source

    Source: Endlessly yield random elements from a given vector. Recall also that
    strings are vectors too, so:
    
    (transduce (take 5) #'string (shuffle "Númenor"))
    => "mNNrú"
    

  • EXTERNAL FUNCTION

    SNOC

    • &OPTIONAL
    • ACC
    • INPUT

    Source

    Reducer: Collect all results as a list, but results are reversed.
    In theory, slightly more performant than `cons' since it performs no final
    reversal.

  • EXTERNAL FUNCTION

    SPLIT

    • TA
    • RA

    Source

    Transducer: Split off a new transducer chain, feeding it each input as well. It
    reduces on its own given RA reducer. The final result is a cons-cell where the
    first value is the result of the original transduction, and the second is that
    of the branch.

  • EXTERNAL FUNCTION

    STEP

    • N

    Source

    Transducer: Only yield every Nth element of the transduction. The first element
    of the transduction is always included.
    
    # Conditions
    
    - `non-positive-integer': when a non-positive integer N is given.
    
    # Examples
    
    (transduce (step 2) #'cons '(1 2 3 4 5 6 7 8 9))
    => (1 3 5 7 9)
    

  • EXTERNAL FUNCTION

    STRING

    • &OPTIONAL
    • ACC
    • INPUT

    Source

    Reducer: Collect a stream of characters into to a single string.

  • EXTERNAL FUNCTION

    TAKE

    • N

    Source

    Transducer: Keep only the first N elements of the transduction.

  • EXTERNAL FUNCTION

    TAKE-WHILE

    • PRED

    Source

    Transducer: Keep only elements which satisfy a given PRED, and stop the
    transduction as soon as any element fails the test.

  • EXTERNAL FUNCTION

    UNCONS

    • REDUCER

    Source

    Transducer: Split up a transduction of cons cells.

  • EXTERNAL FUNCTION

    UNIQUE

    • REDUCER

    Source

    Transducer: Only allow values to pass through the transduction once each.
    Stateful; this uses a hash table internally so could get quite heavy if you're
    not careful.

  • EXTERNAL FUNCTION

    VECTOR

    • &OPTIONAL
    • ACC
    • INPUT

    Source

    Reducer: Collect a stream of values into a vector.

  • EXTERNAL FUNCTION

    WINDOW

    • N

    Source

    Transducer: Yield N-length windows of overlapping values. This is different from
    `segment' which yields non-overlapping windows. If there were fewer items in the
    input than N, then this yields nothing.
    
    # Conditions
    
    - `non-positive-integer': when a non-positive integer N is given.
    

  • EXTERNAL GENERIC-FUNCTION

    CYCLE

    • SEQ

    Source

    Source: Yield the values of a given SEQ endlessly.

  • EXTERNAL GENERIC-FUNCTION

    TRANSDUCE

    • XFORM
    • F
    • SOURCE

    Source

    The entry-point for processing some data source via transductions.
    
    This requires three things:
    
    - A transducer function, or a composed chain of them
    - A reducing function
    - A source
    
    Note: `comp' can be used to chain transducers together.
    
    When ran, `transduce' will pull values from the source, transform them via the
    transducers, and reduce into some single value (likely some collection but not
    necessarily). `transduce' will only pull as many values from the source as are
    actually needed, and does so one at a time. This ensures that large
    sources (like files) don't consume too much memory.
    
    # Examples
    
    Assuming that you've required this library with a local nickname of `t', here's
    how we can filter an infinite source and reduce into a single sum:
    
    (t:transduce (t:comp (t:filter #'oddp)
                         (t:take 1000)
                         (t:map (lambda (n) (* n n))))
                 #'+ (t:ints 1))
    ;; => 1333333000 (31 bits, #x4F790C08)
    
    Note that due to how transducer and reducer functions are composed internally,
    the order provided to `comp' gets applied from top to bottom. In the above
    example, this means that `filter' is applied first, and `map' last.
    
    There are a variety of functions to instead reduce into a collection:
    
    (t:transduce (t:map #'1+) #'t:vector '(1 2 3))
    ;; => #(2 3 4)
    
    Many standard collections can be easily "sourced", including those that aren't
    normally so conveniently traversed like Hash Tables, Property Lists, and lines
    of a file.
    
    ;; Read key-value pairs from a plist and recollect into a Hash Table.
    (t:transduce #'t:pass #'t:hash-table (t:plist `(:a 1 :b 2 :c 3)))
    
    # Custom Sources
    
    Since `transduce' is generic, you can use `defmethod' to define your own custom
    sources. See `sources.lisp' and `entry.lisp' for examples of how to do this.
    
    

  • EXTERNAL MACRO

    ALL

    • PRED

    Source

    Deprecated: Use `allp'.

  • EXTERNAL MACRO

    ANY

    • PRED

    Source

    Deprecated: Use `anyp'.

  • EXTERNAL SOURCE-TRANSFORM

    MAKE-REDUCED

    No documentation provided.
  • EXTERNAL SOURCE-TRANSFORM

    (SETF MAKE-REDUCED)

    No documentation provided.
  • EXTERNAL SOURCE-TRANSFORM

    REDUCED-P

    No documentation provided.
  • EXTERNAL SOURCE-TRANSFORM

    (SETF REDUCED-P)

    No documentation provided.
  • EXTERNAL SOURCE-TRANSFORM

    REDUCED-VAL

    No documentation provided.
  • EXTERNAL SOURCE-TRANSFORM

    (SETF REDUCED-VAL)

    No documentation provided.