JShell in Java 9 - Part 3

 JShell default package: 

Now we will discuss the package related concept, by default how many packages are available in jshell.

Let's see a few examples:

EX.1:-

jshell> Math.sqrt(4)

$4 ==> 2.0 

| created scratch variable $4 : double

EX.2:-

jshell> Math.max(10,20) 

$5 ==> 20

| created scratch variable $5 : int 

EX.3:-

jshell> Math.random() 

$6 ==> 0.6956946870985563

| created scratch variable $6 : double 

• Here you can see we didn't include any package in the jshell still Math class is working because of the java.lang package which is by default available to jshell.

Let's check the whether this code will work in jshell or not?

jshell> ArrayList<String> l=new ArrayList<String>();

l ==> []

As ArrayList, is available in java.util package. So how it is working even we are not importing java.util package in jshell.

By seeing the output it is clear that java.util package is also available to jshell, that's why it has created the empty ArrayList obj.

We can do multiple operations on ArrayList obj like add, remove, getting the size of obj.

Refer the below operation:

There is one question that :

Q1. How many packages are imported by default to jshell?

There are exactly 10 packages available to jshell. How to check which packages are being imported by default to jshell.

To check we need to use /imports command.
Refer the below image:

So, this is all about packages.

Jshell Compiler

Internally jshell having Java compiler which is responsible to check syntax. If any violation in syntax then we will get Compile time error which is exactly the same as normal compile time errors.

Below is the example of compile-time error:

EX.1:

In the Above example, we will get the compile-time error as we didn't declare X and Y so jshell compiler will give compile time error.

EX.2














In this case, jshell compiler will give the compile-time error because we have typed (Sytsem) instead of System and it is not recognized by jshell compiler hence it throws compile time error.



Let's Watch the difference between jshell compiler and java normal compiler:-

In our program, if there is any chance of getting checked exceptions compulsory we required to handle either by try-catch or by throws keyword. Otherwise, we will get a Compile time error.

But in the case of Jshell, jshell itself will take care of these and we are not required to use try-catch or throws. Thanks to Jshell. 

import Java.io.*;
class Test 
 {
    public static void main(String[] args) 
 {
   PrintWriter pw=new PrintWriter("abc.txt"); 
    pw.println("Hello"); 
  }
}

When we run this java program using normal java compiler, then we will get the compile-time error.
but if run the same program in jshell we will not get the compile-time error.
below is the code for jshell

jshell> PrintWriter pw=new PrintWriter("abc.txt");pw.println("Hello");pw.flush(); 
pw ==> Java.io.PrintWriter@e25b2fe 

Conclusions:

1.  From the jshell we can execute any expression, any Java statement.
2.  Most of the packages are not required to import to the Jshell because by default already available to the jshell.
3.  Internally jshell use Java compiler to check syntaxes
4.  If we are not handling any checked exceptions we won’t get any compile time errors, because jshell will takes care.

JShell in Java 9 - Part 2

Getting Started with JShell: 

Q1. How to Start and Stop JShell?

Open the Jshell from the command prompt in verbose mode. Using this command jshell -v Or jshell.

We will discuss further the difference b/w  jshell and jshell -v command.

Refer the below Image:

Q2.How to exit JShell?

Just type this command /exit.

Refer the below Image:

Now its time to discuss the difference b/w with -v and without -v.

-v stands for verbose mode. It means in verbose mode we will get the extra information from the Jshell.

Let's see the example in verbose mode and non-verbose mode.

• Without -v

• With -v
  

Note: If any information displaying on the jshell starts with '|' pipe symbol, it is the information to the programmer from the jshell.

EX:

Refer the below example:

 jshell> System.out.println("Hello World")

 Hello World

  In the above example, we observe that the output does not start with '|' pipe symbol because it is not information from the Jshell.

Note: Terminating semicolons are automatically added at the end of complete snippet by JShell if not entered.

JShell In Java 9 - Part 1

Introduction to the JShell:

This Feature was introduced by Java from Java 9.

Q1.What is Jshell? 

   

Jshell is Java's Shell. It is also known as an Interactive console. 

Jshell is Java's own REPL tool.

REPL  stands for  Read Execute Print Loop.

Refer this image.

By using this tool we can execute Java code snippets and we can get immediate results. 

Using Jshell we can test and execute Java expressions, statements, methods, classes etc. After testing small code we can plug that code in our project.

Before Java 9 we cannot test a small piece of Java code without any Java class. But from Java 9 onwards we can execute the small piece of without having complete class structure.

Its a new thing for java but this kind of tool is already available in the various language like Python, Scala, Swift.

Limitation of Jshell:

•  JShell is not meant for Main Coding. We can use it to test small coding snippets, which can be used in our Main Coding. 
•  JShell is not a replacement of Regular Java IDEs like Eclipse, NetBeans etc.

Java 8 and 9 Enhancement in Interface Part -1

Hello Friends,

In this post, we will take a look at Java 8 and Java 9 Enhancement with respect to Interface.

As we already know in the interface every method is always public and abstract whether we are declaring or not. We can't put the normal method and this is the limitation of interface prior to Java 8 version.

Java 8 version onwards give you the ability to put default method and static method as well.

Now the question comes to mind why we need default method and static method in Interface.
To give you an answer to this question lets take an example.

Step :

Assume we have an interface with the name School and that has one method syllabus.

public interface School {

public void syllabus();

}

Now assume we have ten different class that implement School interface and provide his own implementation for syllabus method.

public class Class1 implements School {

@Override
public void syllabus() {
System.out.println("Class First");
}
}


public class Class2 implements School {

@Override
public void syllabus() {
System.out.println("Class Second");
}
}

and up to class10.

Assume our programming requirement is we have to extend the functionality of this
interface by adding a new method ExtraSub.

public interface School {
public void syllabus();
        public void extraSub(); 
}


Once we added new method extraSub to the interface then all the implementation classes will be
affected and won't be compiled, because every implementation class should implement all
methods of interface.

Hence to overcome this problem in Java 8 default method concept came. which are also known as Defender methods or Virtual Extension Methods.


How to Declare Default Methods inside interfaces :

public interface School {

        default void extraSub(){
System.out.println("Additional subject");
}
public void syllabus();
}

So now Interface default methods are by default available to all implementation classes. Based
on requirement implementation class can ignore these methods or use these default methods directly or can override.

So we can say a default method gives you the ability to add an additional feature without affecting implementation class.

Now assume we have added multiple default method in the interface and some have common functionality so there may be duplicate code to overcome this type of problem Java 9 has private method concept in the interface.

Example : 

public interface School {

         default void extraSub() {
System.out.println("Only for Class 9 and 10");
common();
}

default void extraSub1() {
System.out.println("Only for Class 9 and 10");
common();
}

default void extraSub2() {
System.out.println("Only for Class 9 and 10");
common();
}

default void extraSub3() {
System.out.println("Only for Class 9 and 10");
common();
}

private void common(){
System.out.println("Common code for all the default method");
}
public void syllabus();
}

So in the above example, we have added separately the common code into a private method and call that private method from each default method which required that functionality.

Java serialisation rule

A static variable is not a part of object state and hence it won't participate in serialization due to this declaring static variable as transient there is no use.


Every final variable get value at compile time where as normal variable get value at run-time.


Final variable will be participated in serialisation directly by value hence declaring a final variable as transient there is no impact.


We can serialized any number of file but in which order we have seralized in the same order we have only to deseralized.
i.r serialization and deseralization order must be maintain.


Order of serilalization would be like this,if you are not aware of this

Object o = ois.read()

if(o instanceOf Dog)
Dog d = (Dog)o

if(o instanceOf Cat)
Cat d = (Cat)o

if(o instanceOf Rat)
Rat d = (Rat)o


class Dog {
Dog d = new Dog();
}
Class Cat {
Rat r = new Rat();
}
Class Rat {
 int j =0;
}

In serialization, everything is controlled by JVM, not a programmer, It is also possible to save the total object to the file and it is not possible to save part of the object, which may create performance problem to overcome this problem we should go for externalization.The main advantage of externalization over serialization is everything is taken care by programmer and JVM doesn't have any control based on our requirement we can save total or part of the object which improved system performance.

To provide the externalizable ability for any java class must have to implement the Externalizable interface.That interface has two methods WriteExternal and ReadExternal


The Externalizable(1.1) interface is the child method os serializable(1.1) interface.

Account object has so many properties i want to do serialization  externalization





Case 1: =

If a child doesn't implement serializable we can serialize child class object if parent class implement serializable interface.i.e serializable nature is inherited from parent to child.
hence if the parent is serializable every child is serializable.

Object class doesn't implement serializable interface.

Case 2:=

Even though parent class doesn't implement serializable we can serialize child class if child implements serializable interface.

At the time of serialization, JVM will check if any variable inheriting from a non-serializable parent or not if yes the JVM ignores the original value and save default value to the file.

At the time of deserialization, JVM will check if any parent class non-serializable or not if any parent is non-serializable then JVM will execute instance control flow for every and share instance variable to the current object

Every non-serializable should compulsorily contain no argument constructor it may be default constructor generator to compile or customized constructor explicitly provided by programmer otherwise we will get a runtime constructor invalid class exception.

Scala Serialization

Serialization: The process of writing state of an object into a file, Basically it is a process of converting an object from Java supported form into file supported form or network supported form.

Object means binary data to write into the file we need a stream.

By using FileOutputStream and ObjectOutputStream classes we can achieve serilization.

Deserialization: The process of reading the state of the object from the file is called Deserialization, Basically, it is a process of converting an object from file supported form or network supported into form Java supported form.



By using FileInputStream and ObjectInputStream classes we can achieve deserilization.



import java.io._

@SerialVersionUID(123L)
class Dog(var i:Int= 10,var j:Int= 20) extends  Serializable{

}


object Dog{
  def main(args:Array[String]){
    var d1 = new Dog()
 
    val oos = new ObjectOutputStream(new FileOutputStream("dog.ser"))
    oos.writeObject(d1)
    oos.close
 
 
    val ois = new ObjectInputStream(new FileInputStream("dog.ser"))
    val d2 = ois.readObject.asInstanceOf[Dog]
    ois.close
    println(d2.i)
    println(d2.j)
 
  }
}



We can serialize the only serializable object. To make an object serializable we need to implement serializable interface.It is a marker interface and available in the java.io package.
You will get a runtime exception Not serializable exception if you haven't implemented serializable interface.

Transient : if we use a Transient variable in the class that variable is not a part of the serialization. JVM ignore the original value and assign a default variable to that variable.

So we can say transient means not to participate in serialization.

Apache Spark for Beginners

What is Apache Spark?

Apache Spark is a general-purpose & lightning fast an open-source cluster computing system that provides high-level API in Java, Scala, Python and R. It can access data from HDFS, Cassandra, HBase, Hive, Tachyon, and any Hadoop data source. And run in Standalone, YARN and Mesos cluster manage

Apache Spark provide three different unit of abstraction to work with.In this post we will provide you basic understanding of each of them and also do same hands-on so that you will get better understanding.

1. RDD
2. DataFrame
3. Dataset

RDD(Resilient Distributed Dataset) :

RDD stands for Resilient Distributed Dataset. What does it means.

Resilient - Immutable and fault tolerant that you can't modified once created
Distributed - RDD is distributed over cluster.(RDD are divided into small chunks are called partitions that is distributed across the cluster)

Dataset - That holds Data

So, An RDD is the resilient(or immutable),partitions, distributed collection of data. 

Most basic data unit in Spark upon which all Operations are performed. In RDD intermediate results is stored in Memory.

When to use RDD?

• If your data is unstructured.
• If you want to work with functional approach.
• If you don't care about imposing schema.

How to create RDD?

Common ways to build the RDD:

• Using SparkContext.parallelize that used existing collection. Assuming sc is spark context object.

             val num = Array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 ,15)

             val rdd = sc.parallelize(num)

             

             

• By using external files or source of data(like txt file,csv file,hdfs)

             val data = sc.textFile("data.txt") 

• SparkContext.makeRDD

             val rdd = sc.makeRDD(0 to 1000)

Operations(or in simple word operators) on RDD

RDD support 2 type of operation transformations and actions

Transformation – Transformations are functions that take a RDD as the input and produce one or many RDDs as the output. They do not change the input RDD as we know RDD is immutable.

Note: - Every transformations return a new RDD and keep a pointer to the parent RDD and the result RDD will always be different from their parents and can be smaller (e.g. filter, count, distinct, sample), bigger (e.g. flatMap, union, cartesian) or the same size (e.g. map).

All transformations are lazy in spark, i.e. Spark doesn't execute it immediately instead it creates a lineage.

Lineage is just keep a track of all the transformation has to be applied on that RDD.

There are two types of transformations.

• Narrow Transformations
• Wide Transformations

Narrow Transformation - RDD operations like map, union, filter can operate on a single partition and map the data of that partition to resulting single partition. These kind of operations are known as Narrow operations. In Narrow operation no need to distribute the data across the partitions.

Wide Transformation - RDD operations like groupByKey, distinct, join may require to map the data across the partitions in new RDD. These kind of operations which maps data from one to many partitions are referred as Wide operations.Generally Wide operations distribute the data across the partitions. Wide transformation is most costly than narrow operations due to data shuffling.

List of common transformation supported by Spark

Action – Spark Action returns final result to driver program or save it to the external location. Action trigger execution of RDD transformation as we know transformation is lazy.Simply we can say action evaluates lineage graph.

toDebugString() function gives information about  RRD lineage.

List of common action supported by Spark.

Several File Formats In spark

How to read and write using spark

1. TEXT File
2. CSV File
3. JSON File
4. PARQUET File
5. ORC File
6. AVRO File
7. SEQUENCE File

Reading and Writing several file formats in spark 2.0

After Rdd and Dataframe another abstraction is introduce in spark 2.0 called Dataset is a super set of Dataframe. 

In earlier version of spark, Spark Context was the entry point. Now from spark 2.0 onward Spark Session is the main entry point for Dataset and Dataframe.

SparkSession internally has spark context and is a combination of HiveContext,SQLContext

and is available with name spark in spark-shell

Creating SparkSession

val spark = SparkSession

                     .builder

                     .master("local")

                     .appName("Spark Job")

                     .getOrCreate()

Reading and Writing several file formats in spark 1.6