0
$\begingroup$

I'm having problems reading the data from event hub. questions :

  1. what am i doing wrong in connecting to event hub using databricks?
  2. is the problem with the message? or is it in the script?
  3. is the error on the stream analytics indicate there is a problem with the message?

The project is in Azure environment and involves these resources : EventHub , Databricks and Stream analytics.

First i've published events to the Eventhub, and am trying to read the data using Databricks.

When trying to read the message in Stream analytics i get this error :

While sampling data, no data was received from '1' partitions.
Will try to sample previous events.

while running the databricks code i seem to establish a connection to the eventhub using this script:

%scala
val namespaceName = "###-eventhubs"
val eventHubName  = "###-hub"
val sasKeyName    = "###ExportPolicy"
val sasKey        = "###SBE="
val connectionString       = new com.microsoft.azure.eventhubs.ConnectionStringBuilder()
                    .setNamespaceName(namespaceName)
                    .setEventHubName(eventHubName)
                    .setSasKeyName(sasKeyName)
                    .setSasKey(sasKey)

val customEventhubParameters = 
  EventHubsConf(connectionString.toString())
  .setMaxEventsPerTrigger(5)

val ehConf = EventHubsConf(connectionString.toString())
            .setConsumerGroup("###")




and get a result : enter image description here

when adding this script to read the message the script don't terminate

val incomingStream = spark.readStream.format("eventhubs").options(customEventhubParameters.toMap).load()

incomingStream.writeStream.outputMode("append").format("console").option("truncate",false).start().awaitTermination()

enter image description here

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.