Giter Club home page Giter Club logo

scala-spark-mail's Introduction

Scala-Spark-Mail

Scala-Spark-Mail is the utility which uses to send Spark job failure and Success message in a mail. It uses smtp protocol to send mail and primarily based on javax API. It is very easy to use, you just need to create a object and provide a complete path of Configuratin file where you need to mentioned all properties mentioned in below application.conf file.

https://github.com/NikhilSuthar/ScalaMail/blob/master/src/main/resource/application.conf

Compiled Version

  • Scala "2.11.12"
  • SBT 1.2.8

How to Use in Program

  • Download complete Jar file directly from below path or else clone it and compile it using command sbt clean assembly

    https://github.com/NikhilSuthar/ScalaMail/tree/master/Jar

  • Add this Jar dependencies in Spark Project (in build.sbt) like below:

    "com.spark.mail" %% "Email" % "0.0.1" from "file:///<path of Jar>/Scala_Spark_Mail.jar"

    or

    Add module dependencies in intelliJ or Eclipse

    click here for details about adding unmannaged dependecny.

  • Edit configuration file and add below properties

      email {
              email_host="smtp.gmail.com"
              email_port="587"
              email_username= "userName"
              email_password="welcome123"
              email_recipient="<list of receipent mail with comma separated>"
              email_auth ="true"
              email_ssl_enable="false"
              email_starttls_enable="true"
              }
      		
      spark {
        appName = "Spark_Test_Job"
      }
    
  • Import method in Spark Scala Object or Class

  • Initialize Email Object as below

    val msg = "Spark Job failed" //you can initialize msg as exception of failure

    val Emailobj = new Email(<path of Conf file>)

    Emailobj.sendMail(msg)

    sendMail Method

    sendMail method comes with four parameters, in which three parameters are optional. Please find below details:

    • sendMail(message, SparkApplicationId, MailSubjectLine,MailType,defaultMessage,ListOfAttachments)

      • sendMail(message,SparkApplicationId,"","F"): Send message as Failure with subject as "Alert:Spark SparkAppName Job.".
      • sendMail(message,SparkApplicationId,MailSubjectLine,"F"): Send message as Failure with subject as MailSubjectLine.
      • sendMail(message,SparkApplicationId,"","S"): Send message as Success with subject as "Alert:Spark SparkAppName Job.".
      • sendMail(message,SparkApplicationId,MailSubjectLine,"S"): Send message as Success with subject as MailSubjectLine.
      • sendMail(message,SparkApplicationId): Send message as Success with subject as "Alert:Spark SparkAppName Job.".
      • sendMail(message): Send message as Success with subject as "Alert:Spark SparkAppName Job.". It can use for Scala Job without Spark.
    • Default Value

      • SparkApplicationId = blank ("")
      • MailSubjectLine = "Alert:Spark SparkAppName Job."
      • MailType = "S"
        • F - For Failure
        • S or Blank("") - For Success
        • R - For Report
      • defaultMessage = Message that display at header of message.
        • default value are:
          • When MailType is "F" - "Spark Job has been Failed On Date Time . Failed due to below reason:"
          • When MailType is "R" - "Please find below Report:"
          • When MailType is "" Or other than "F" & "R" - No Header
      • ListOfAttachments = Complete paths of all attachment files separated by Semi Colon (;).
        • default value is blank.
        • It take String of complete paths of all files separated by Semi Colon (;) and Name separated by Comma within File path as below.
        • When pass value as - "FilePath1;FilePath2" or "FilePath" - It send all files with Name as file system.
        • When pass value as - "Name1,FilePath1;Name2,FilePath2" or "Name,FilePath" - It send all files with respective Name separated by comma (,) with File Path.

    Sample Spark Code

      object TestSPark {
          def main(args: Array[String]): Unit = {
      	try{
      	      /* Logic of Spark Program */
      	     val Emailobj = new Email(<path of Conf file>)
      	     Emailobj.sendMail(msg,appId)
      	    } catch {
      		    case e: Exception => 
      			    val msg = e.toString
      			    val Emailobj = new Email(<path of Conf file>)
      			    Emailobj.sendMail(msg,appId,"","F")
      		       }
      	     }
            }
    

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.