MemoryStreamBase -- Base Contract for Memory Sources¶
MemoryStreamBase
is the <
[[contract]] .MemoryStreamBase Contract [cols="30m,70",options="header",width="100%"] |=== | Method | Description
| addData a| [[addData]]
[source, scala]¶
addData( data: TraversableOnce[A]): Offset
| logicalPlan a| [[logicalPlan]]
[source, scala]¶
logicalPlan: LogicalPlan¶
|===
[[implementations]] .MemoryStreamBases [cols="30,70",options="header",width="100%"] |=== | MemoryStreamBase | Description
| ContinuousMemoryStream | [[ContinuousMemoryStream]]
| MemoryStream | [[MemoryStream]] MicroBatchReader for Micro-Batch Stream Processing
|===
=== [[creating-instance]] Creating MemoryStreamBase Instance
MemoryStreamBase
takes the following to be created:
- [[sqlContext]]
SQLContext
NOTE: MemoryStreamBase
is a Scala abstract class and cannot be <
=== [[toDS]] Creating Streaming Dataset -- toDS
Method
[source, scala]¶
toDS(): Dataset[A]¶
toDS
simply creates a Dataset
(for the <
=== [[toDF]] Creating Streaming DataFrame -- toDF
Method
[source, scala]¶
toDF(): DataFrame¶
toDF
simply creates a Dataset
of rows (for the <
=== [[internal-properties]] Internal Properties
[cols="30m,70",options="header",width="100%"] |=== | Name | Description
| attributes a| [[attributes]] Schema attributes of the <Seq[AttributeReference]
)
Used when...FIXME
| encoder a| [[encoder]] Spark SQL's ExpressionEncoder
for the data
Used when...FIXME
|===