Create

Create is a utility to create in-memory collections (as Values or TimestampedValues root PTransforms that produce the values when executed).

Create utility is especially useful during testing.

empty Utility

Values<T> empty(
  Coder<T> coder)
Values<Row> empty(
  Schema schema)
Values<T> empty(
  TypeDescriptor<T> type)

empty creates an empty Values root PTransform.

of Utility

Values<T> of(
  Iterable<T> elems)
Values<KV<K, V>> of(
  Map<K, V> elems)
Values<T> of(
  T elem,
  T... elems)

of creates a Values root PTransform with the elements given.

ofProvider Utility

OfValueProvider<T> ofProvider(
  ValueProvider<T> provider,
  Coder<T> coder)

timestamped Utility

TimestampedValues<T> timestamped(
  Iterable<T> values,
  Iterable<Long> timestamps)
TimestampedValues<T> timestamped(
  Iterable<TimestampedValue<T>> elems)
TimestampedValues<T> timestamped(
  TimestampedValue<T> elem,
  TimestampedValue<T>... elems)

timestamped creates a TimestampedValues with the elements given.

Demo

import org.apache.beam.sdk.Pipeline
val p = Pipeline.create()

import org.apache.beam.sdk.transforms.Create
val nums = Create.of(0, 1, 2, 3, 4, 5)

scala> :type nums
org.apache.beam.sdk.transforms.Create.Values[Int]

val input = p.apply("Input Numbers", nums)

scala> :type input
org.apache.beam.sdk.values.PCollection[Int]

Values Root PTransform

Values<T> is a root PTransform that produces a PCollection from a set of in-memory objects (of type T).

PTransform<PBegin, PCollection<T>>

Values takes the following to be created:

  • Elements (Iterable<T>)

  • Optional Coder (Optional<Coder<T>>)

  • Optional TypeDescriptor (Optional<TypeDescriptor<T>>)

Values is created using Create.of and Create.empty utilities.

Values is a static class of [] utility.

TimestampedValues Root PTransform

TimestampedValues<T> is…​FIXME