I'm having problems reading the data from event hub. questions :
- what am i doing wrong in connecting to event hub using databricks?
- is the problem with the message? or is it in the script?
- is the error on the stream analytics indicate there is a problem with the message?
The project is in Azure environment and involves these resources : EventHub , Databricks and Stream analytics.
First i've published events to the Eventhub, and am trying to read the data using Databricks.
When trying to read the message in Stream analytics i get this error :
While sampling data, no data was received from '1' partitions.
Will try to sample previous events.
while running the databricks code i seem to establish a connection to the eventhub using this script:
%scala
val namespaceName = "###-eventhubs"
val eventHubName = "###-hub"
val sasKeyName = "###ExportPolicy"
val sasKey = "###SBE="
val connectionString = new com.microsoft.azure.eventhubs.ConnectionStringBuilder()
.setNamespaceName(namespaceName)
.setEventHubName(eventHubName)
.setSasKeyName(sasKeyName)
.setSasKey(sasKey)
val customEventhubParameters =
EventHubsConf(connectionString.toString())
.setMaxEventsPerTrigger(5)
val ehConf = EventHubsConf(connectionString.toString())
.setConsumerGroup("###")
when adding this script to read the message the script don't terminate
val incomingStream = spark.readStream.format("eventhubs").options(customEventhubParameters.toMap).load()
incomingStream.writeStream.outputMode("append").format("console").option("truncate",false).start().awaitTermination()