Annex II.1. Configuring a Secure FTP on Our Linux Server

1. Installing vsftpd

$ sudo apt-get install vsftpd

2. Opening the Firewall

$ ufw status

If the firewall is active, you will have to open some ports. If it is inactive, you won’t have any problem with the FTP server (but there might be a problem with the security of your server). If you need to open ports in your firewall for the FTP server: 20, 21 for FTP, and 990 to enable TLS.

$ sudo ufw allow 20,21,990/tcp

Additionally, we will need to open some ports for passive mode, for example, from 40000 to 50000.

$ sudo ufw allow 40000:50000/tcp

3. Preparing the User Directory

Now, we are going to create a specific user for FTP and configure its home directory to be secure, with no chance to exit to the main directories of the system.

First, we add a user for FTP connections.

$ sudo adduser maricheloftp

You will be asked for a password and some other information. We only need to enter the password; the other information can remain empty.

Vsftpd jails local users in their home directory, and it may not be writable by the user when using shell connections instead of FTP. To avoid this and continue to secure our FTP connections, we are going to create a specific directory for FTP connections where the user, through FTP, will be unable to exit this directory.

$ sudo mkdir /home/maricheloftp/ftp
$ sudo chown nobody:nogroup /home/maricheloftp/ftp
$ sudo chmod a-w /home/maricheloftp/ftp

If we verify the permissions:

$ sudo ls -la /home/maricheloftp/ftp

dr-xr-xr-x 3 nobody       nogroup      4096 Feb 21 17:24 .
drwxr-xr-x 3 maricheloftp maricheloftp 4096 Feb 21 18:48 ..

Now, we are going to create a directory for uploading files.

$ sudo mkdir /home/maricheloftp/ftp/files
$ sudo chown maricheloftp:maricheloftp /home/maricheloftp/ftp/files

$ sudo ls -la /home/maricheloftp/ftp

dr-xr-xr-x 3 nobody       nogroup      4096 Feb 21 17:24 .
drwxr-xr-x 3 maricheloftp maricheloftp 4096 Feb 21 18:48 ..
drwxr-xr-x 2 maricheloftp maricheloftp 4096 Feb 21 18:57 files

And here, we can create a file to test our server.

$ echo "test file" | sudo tee /home/maricheloftp/ftp/files/test.txt

4. Configuring vsftpd

$ sudo nano /etc/vsftpd.conf

Here, you have to change or uncomment the following lines:

anonymous_enable=NO
local_enable=YES
write_enable=YES
chroot_local_user=YES

And add these ones (to enable passive FTP ports):

pasv_min_port=40000
pasv_max_port=50000

and these ones:

user_sub_token=$USER
local_root=/home/$USER/ftp

These last two lines will ensure that local users, when connecting through FTP, will access directly to the ftp directory and not to their home directory.

To allow FTP access only to some users, we add these lines:

userlist_enable=YES
userlist_file=/etc/vsftpd.userlist
userlist_deny=NO

We create the file vsftpd.userlist with the names of the users that will be allowed to use FTP.

$ echo "maricheloftp" | sudo tee -a /etc/vsftpd.userlist

Restart the service.

$ sudo service vsftpd restart

To check if the service is running okay, we can see its status with:

$ sudo service vsftpd status

Enter an editor with the information; to exit, press q.

Testing FTP Access

You can test the FTP access with the ftp command available in Linux and Windows. For Mac, you can use Finder and the option Go and Connect to a Server.

$ ftp maricheloftp@ftp.marich

elo.es

$ ftp ftp.marichelo.es

You will be asked for the password, and you can list with ls and change directory with cd. If you try to connect without inserting a user or using a valid user but not the one included in the vsftpd.userlist file, the connection will be refused.

Now, you can check your Java sources to connect to an FTP server.

6. Securing our FTP server

FTP does not encrypt data transactions, including user credentials, to avoid this hole in security we are going to enable TLS/SSL to provide encryption. First, we have to create the SSL certificates in our server to use them with vsftpd. Here we use openssl to create a certificate valid for 1 year and both the private key and the certificate will be located in the same file. This is only one instruction, only one line.

$ sudo openssl req -x509 -nodes -days 365 -newkey 
rsa:2048 -keyout /etc/ssl/private/vsftpd.pem -out 
/etc/ssl/private/vsftpd.pem

You will be asked about your address information that will be incorporated in your certificate request.

Once you have created the certificates, we will open the file vsftpd.conf again. We will comment these lines:

#rsa_cert_file=/etc/ssl/certs/ssl-cer-snakeoil.pem
#rsa_private_key_file=/etc/ssl/private/ssl-cert-snakeoil.key

And we will add these ones:

rsa_cert_file=/etc/ssl/private/vsftpd.pem
rsa_private_key_file=/etc/ssl/private/vsftpd.pem

Now, we are going to force the use of SSL always in our FTP server. Change ssl_enable to YES:

ssl_enable=YES

Next, add the following lines to explicitly deny anonymous conections over SSL and require SSL for both data transfer and logins:

allow_anon_ssl=NO
force_local_data_ssl=YES
force_local_logins_ssl=YES

With the following lines we will configure our server to use TLS:

ssl_tlsv1=YES
ssl_sslv2=NO
ssl_sslv3=NO

And finally, we will add these 2 options to don’t require reuse of ssl because it can break many FTP clients and to use key lenghts equal or greater than 128 bits:

require_ssl_reuse=NO
ssl_ciphers=HIGH

Now we will close the configuration file and restart the service. Now, if we try to to connect with the command ftp we will have a message saying that non-anonymous sessions must use encryption.

To try it now we will need a FTP client that supports TLS as FileZilla or our Java program using the class FTPSClient.

Unit 3. Basic Client-Server Communications

3.3. FTP connections. FTP clients

In this section we are going to see how to connect to a FTP server using a Java application. So, we need to implement an FTP client for this purpose. To access a FTP server through Java we’ll use Apache’s FTPClient class which is included in Apache’s Commons Net library. To do this, we will have to create a Maven or Gradle project in order to add this library in the dependencies, or download the corresponding JAR and include it in our project.

3.3.1. Connection

The most simple way to connect to an existing FTP server through FTP protocol (you can connect to an FTP server through HTTP protocol also) is opening a connection like this:

 1FTPClient ftp = new FTPClient();
 2try {
 3    // Important (before connecting)
 4    ftp.setControlEncoding("UTF-8"); 
 5	ftp.connect("172.16.208.128");
 6	System.out.print(ftp.getReplyString());
 7    // If the server is in another network...
 8	ftp.enterLocalPassiveMode(); 
 9
10	if(!FTPReply.isPositiveCompletion(ftp.getReplyCode())) {
11		ftp.disconnect();
12		System.err.println("Error connecting to FTP");
13	}
14} catch (IOException e) {
15} finally {
16	if (ftp.isConnected()) {
17		try {
18			ftp.disconnect();
19		} catch (IOException ioe) {}
20	}
21}

As you can see, we open a connection specifying the server IP or domain name (localhost if it’s running on the same machine). Then we print the response from the server and check the reply code (indicating an error or success) using the class FTPReply to know if the connection was successful or not.

3.3.2. File listing

To list the contents of a directory, first we’ll have to log in, and then we can use the listFiles() method that returns an array of FTPFile objects, representing each one of them a file (or subdirectory, or link) inside the directory:

1if(ftp.login("arturo", "arturo")) {
2    FTPFile[] files = ftp.listFiles();
3    for(FTPFile file: files) {
4		String type = file.isDirectory()?
5   			"Directory":file.isSymbolicLink()?
6   					"Link":"File";
7        System.out.println(type + " -> " + file.getName());
8    }
9}

3.3.3. Uploading files

Before uploading a file to the server we must know if that file is in binary or text format and set the FTP default file type to be transferred previous to send it. This is accomplished with setFileType(int type) method. Valid values are FTP.ASCII_FILE_TYPE (default) or FTP.BINARY_FILE_TYPE (recommended if you don’t know the type of a file).

Other useful methods are changeWorkingDirectory(String pathname) that changes the current working directory in the FTP server and changeToParentDirectory() which changes to the parent directory.

 1public static void uploadFile(boolean isText, String filePath, 
 2    String nameInServer) {
 3    // This is a method we have created to open a connection
 4	if(!connect()) { 
 5		System.err.println("Cannot upload file, error connecting!");
 6		return;
 7	}
 8		
 9	try(FileInputStream in = new FileInputStream(filePath)) {
10		ftp.setFileTransferMode(
11			isText?FTP.ASCII_FILE_TYPE:FTP.BINARY_FILE_TYPE);
12        if(!ftp.storeFile(nameInServer, in)) {
13            System.err.println("Error uploading file " + filePath +
14                " (" + ftp.getReplyString() + ")");
15        } else {
16            System.out.println("File " + filePath +
17            " uploaded with name " + nameInServer);	
18        }
19    } catch (IOException e) {
20        System.err.println("Error uploading file " + filePath
21                + e.getMessage());
22    }
23}

3.3.4. Downloading files

To download a file is a very similar process but using a FileOutputStream on the local filename and the method retrieveFile.

 1public static void downloadFile(boolean isText, String nameInServer, 
 2    String nameLocal) {
 3	if(!connect()) {
 4		System.err.println("Cannot download file, error connecting!");
 5		return;
 6	}
 7	try(FileOutputStream out = new FileOutputStream(nameLocal)) {
 8		ftp.setFileTransferMode(isText? 
 9            FTP.ASCII_FILE_TYPE:FTP.BINARY_FILE_TYPE);
10        if(!ftp.retrieveFile(nameInServer, out)) {
11            System.err.println("Error downloading file " + nameInServer +
12                " (" + ftp.getReplyString() + ")");
13        } else {
14            System.out.println("File " + nameInServer +
15                " downloaded with name " + nameLocal);	
16        }
17	} catch (IOException e) {
18        System.err.println("Error downloading file " + nameInServer +
19            e.getMessage());
20	}
21}

3.3.5. Other operations

There are other useful operations that can be done from the FTP client, such as:

• rename(String from, String to): Changes a remote file’s name.
• deleteFile(String pathname): Deletes a remote file.
• removeDirectory(String pathname): Deletes a directory, only if it’s empty.
• makeDirectory(String pathname): Creates a new subdirectory.
• printWorkingDirectory(): Gets the name of the current working directory.
• listNames() Gets only the names of the list of files inside the current directory.

Exercise 6

Create a JavaFX application called FTPManager. It will be a simple FTP client, which you’ll use to connect to a server (can be localhost or a virtual machine IP), list its contents on a ListView and do several actions. The application aspect should be more or less like this:

As you can see, once you enter a server address, a login name and password, and click the Connect button, the application will list the working directory contents. The other buttons will perform the following actions:

• Upload: Will open a FileChooser dialog to select a file that will be uploaded to the FTP server (current directory). Dowload/Enter: Only active when something in the list is selected. Two situations can happen
  • If the selected item is a directory, the application will change to that directory listing its files.
  • If it’s a normal file, the application will open a DirectoryChooser dialog to select a local directory and download the file there. Go up: Will change to the parent directory and list its contents. Delete: Only active when something in the list is selected. Will delete the file.

There will be a label where you’ll show every action’s result, whether it’s a success or a failure. When it’s needed, refresh the list’s contents.

3.4. FTPS Connections (FTP Secure)

FTPS adds layers of security to standard FTP connections through the use of SSL (Secure Sockets Layer) or TLS (Transport Layer Security). To access an FTPS server through Java, we will use Apache’s FTPSClient class from the Apache Commons Net library.

3.4.1. Connection

Connecting to an FTPS server is similar to connecting to an FTP server but using FTPSClient instead of FTPClient:

 1ftps = new FTPSClient("TLS",false);
 2try {
 3    ftps.setControlEncoding("UTF-8");
 4    //ftps.connect("54.38.240.72");
 5    ftps.connect("ftp.marichelo.es");
 6    ftps.login("maricheloftp", "marichelo");
 7
 8    if (!FTPReply.isPositiveCompletion(ftps.getReplyCode())) {
 9        ftps.disconnect();
10        System.err.println("Error connecting to FTPS");
11    } else {
12        ftps.enterLocalPassiveMode();
13        ftps.execPROT("P");
14        System.out.println("Connected");
15        //testing chroot
16        System.out.println(ftps.printWorkingDirectory());//wich directory we are connected
17        ftps.cdup(); // cd.. trying to exit our home directory
18        System.out.println(ftps.printWorkingDirectory());
19
20        FTPFile[] files = ftps.listFiles();  //listing the files of the working directory
21        for(FTPFile file: files) {
22            String type = file.isDirectory()?
23                    "Directory":file.isSymbolicLink()?
24                    "Link":"File";
25            System.out.println(type + " -> " + file.getName());
26        }
27    }
28} catch (IOException e) {
29    e.printStackTrace();
30} finally {
31    if (ftps.isConnected()) {
32        try {
33            ftps.logout();
34            ftps.disconnect();
35        } catch (IOException ex) {
36            ex.printStackTrace();
37        }
38    }
39}

Both uploading and downloading files are similar to FTPClient but using FTPSClient class instead.

3.5. SFTP Connections (SSH File Transfer Protocol)

SFTP is another protocol that provides a secure way to transfer files over an encrypted connection. To implement an SFTP client in Java, we can use the JSch library from com.jcraft.

3.5.1. Connection

Here’s a basic example of how to connect to an SFTP server using JSch:

 1import com.jcraft.jsch.JSch;
 2import com.jcraft.jsch.Session;
 3
 4JSch jsch = new JSch();
 5try {
 6    Session session = jsch.getSession("username", "yourserver.com", 22);
 7    session.setPassword("password");
 8
 9    // Configuration to not validate the host key
10    session.setConfig("StrictHostKeyChecking", "no");
11
12    session.connect();
13
14    // Connection established
15    session.disconnect();
16} catch (Exception e) {
17    e.printStackTrace();
18}

3.5.2. Listing Files, Uploading, and Downloading

For operations like listing files, uploading, and downloading, you’ll need to use the ChannelSftp class from JSch. Here’s a basic outline:

 1import com.jcraft.jsch.Channel;
 2import com.jcraft.jsch.ChannelSftp;
 3import com.jcraft.jsch.JSch;
 4import com.jcraft.jsch.Session;
 5
 6// Establish connection
 7Channel channel = session.openChannel("sftp");
 8channel.connect();
 9ChannelSftp sftpChannel = (ChannelSftp) channel;
10
11// List files
12Vector<ChannelSftp.LsEntry> list = sftpChannel.ls("/path/to/directory");
13for (ChannelSftp.LsEntry entry : list) {
14    System.out.println(entry.getFilename());
15}
16
17// Upload file
18sftpChannel.put("localfilepath", "remotefilepath");
19
20// Download file
21sftpChannel.get("remotefilepath", "localfilepath");
22
23// Close connection
24sftpChannel.exit();
25session.disconnect();

Unit 4. Developing and Accessing Services

4.6. Basic service access from Java

Nowadays, most applications rely on web services to access remote and centralized data stored in a remote server through the World Wide Web. A web service is a technology which uses a series of standards in order to communicate an application running in the server (using any technology like PHP, Node, Ruby, Java, .NET, …) with a client application that can be running in any device and be written also in any language. In this part we’ll mainly focus in accessing REST web services using Java as a client, relying on a server side implemented in Node.js from previous sections.

4.6.1. Opening and reading an HTTP connection

There are many ways to connect to a web via the HTTP protocol in Java. For instance, we can use the native classes derived from URLConnection, or an external library that simplifies the job.

Because one of the main uses of Java is for Android development, we’ll look at what’s recommended there. There’s a library called Apache Http Client, that was included in the Android libraries but it’s not supported anymore (and even if you still can use it, it’s not recommended). The recommended option is to use URLConnection class and its derivatives like HttpURLConnection and HttpsURLConnection.

4.6.1.1. Using URLConnection

This is the most low-level connection method, meaning that the programmer will be controlling every aspect of the connection but in contrast the resulting code will be larger and uglier. It’s recommended in Android because, if used properly, it’s the faster and the least memory, processor (and battery) consuming method.

The URL class is used to represent the remote resource in the World Wide Web we’ll be accessing.

1URL google = new URL("http://www.google.es");

This object will return a URLConnection object when we connect to it.

1URLConnection conn = google.openConnection();

To get the response body (content), the connection provides an InputStream for that purpose. It’s also recommended to retrieve the charset encoding from the response headers, in order to read everything (like accents) properly. We can use this static method for this purpose:

1// Get charset encoding (UTF-8, ISO,...)
2public static String getCharset(String contentType) {
3    for (String param : contentType.replace(" ", "").split(";")) {
4        if (param.startsWith("charset=")) {
5            return param.split("=", 2)[1];
6        }
7    }
8    return null; // Probably binary content
9}

This can be a basic way of connecting to a URL and gettint its contents:

 1public static void main(String[] args) {
 2    BufferedReader bufInput = null;
 3    try {
 4        URL google = new URL("http://www.google.es");
 5        URLConnection conn = google.openConnection();
 6        
 7        String charset = getCharset(conn.getHeaderField("Content-Type"));
 8        
 9        bufInput = new BufferedReader(
10                new InputStreamReader(conn.getInputStream(), charset));
11        
12        String line;
13        while((line = bufInput.readLine()) != null) {
14            System.out.println(line);
15        }
16    } catch (MalformedURLException e) {
17        ...
18    } catch (IOException e) { 
19        ...
20    } finally {
21        if(bufInput != null) {
22            try {
23                bufInput.close();
24            } catch (IOException e) {...}
25        }
26    }
27}

4.6.1.2. HttpURLConnection and following redirections

The class HttpURLConnection provides additional methods like following redirections automatically or getting the response code (such as 404 for “Not Found”). To get an HttpURLConnection and follow redirections automatically you should call this method:

1URL sanvi = new URL("http://iessanvicente.com");
2HttpURLConnection conn = (HttpURLConnection)sanvi.openConnection();
3conn.setInstanceFollowRedirects(true);

It doesn’t always work. For example, it may return code 301 (Moved Permanently) and thus you will not be redirected to the new location automatically. You can check what the response (and its headers) is by using available methods:

1System.out.println(conn.getResponseCode());
2System.out.println(conn.getResponseMessage());
3System.out.println(conn.getHeaderFields());

With this information, we could manage manually these redirections (with the risk of falling into a redirection loop), even if there are many, like this:

1URL url = new URL("http://iessanvicente.com");
2HttpURLConnection conn;
3do {
4    conn = (HttpURLConnection)url.openConnection();
5    if(conn.getResponseCode() == 301) {
6        url = new URL(conn.getHeaderField("Location"));
7    }
8} while(conn.getResponseCode() == 301);

Exercise 8

Create a console Java application named LinkSearch that will ask you for an address and print all the links (<a>) detected in the response. If you want, it’s a good idea to create an auxiliary class that extends from BufferedReader as we saw on Unit 1, to only filter those links from the output. This is part of the output that https://iessanvicente.com should show:

4.6.2. Basics of web service access

To access a REST web service we need a URL (which represents a resource being accessed), and an operation (GET, POST, PUT, DELETE) to do with that resource, along with additional data needed for the operation.

The simplest operation is GET, that is usually used for searching and getting information about something that already exists. In a GET operation, data (if necessary) is sent in the URL in two possible ways:

• http://domain/resource?data1=value1&data2=value2.
• http://domain/resource/value1/value2

This is a really basic Express service that will read two numbers passed in the url (GET) and will print (response) the result of that sum:

1app.get('/sum/:n1/:n2', (req, res) => {
2    let result = parseInt(req.params.n1) + parseInt(req.params.n2)
3    res.send("" + result);
4});

And this is how we can call it from Java and obtain the result:

 1private static String getSumFromService(int n1, int n2) {
 2    BufferedReader bufInput = null;
 3    String result;
 4    try {
 5        URL google = new URL("http://localhost/services/sum/" 
 6            + n1 + "/" + n2);
 7        URLConnection conn = google.openConnection();
 8        
 9        bufInput = new BufferedReader(
10                       new InputStreamReader(conn.getInputStream()));
11        result = bufInput.readLine();
12    } catch (IOException e) {
13        return "Error";
14    } finally {
15        if(bufInput != null) {
16            try {
17                bufInput.close();
18            } catch (IOException e) { return "Error"; }
19        }
20    }
21    
22    return result == null?"Error":result;
23}
24
25public static void main(String[] args) {
26    System.out.println(getSumFromService(3, 5));
27}

4.6.2.1. ServiceUtils class

In order to wrap all the code needed to connect to a web service and send/receive information to/from it, we are going to create our own class. We call it ServiceUtils, and its code is:

 1public class ServiceUtils {
 2
 3    private static String token = null;
 4
 5    public static void setToken(String token) {
 6        ServiceUtils.token = token;
 7    }
 8
 9    public static void removeToken() {
10        ServiceUtils.token = null;
11    }
12
13    public static String getCharset(String contentType) {
14        for (String param : contentType.replace(" ", "").split(";")) {
15            if (param.startsWith("charset=")) {
16                return param.split("=", 2)[1];
17            }
18        }
19
20        return null; // Probably binary content
21    }
22
23    public static String getResponse(String url, String data, 
24        String method) {
25
26        BufferedReader bufInput = null;
27        StringJoiner result = new StringJoiner("\n");
28        try {
29            URL urlConn = new URL(url);
30            HttpURLConnection conn = 
31                (HttpURLConnection) urlConn.openConnection();
32            conn.setReadTimeout(20000 /*milliseconds*/);
33            conn.setConnectTimeout(15000 /* milliseconds */);
34            conn.setRequestMethod(method);
35
36            conn.setRequestProperty("Host", "localhost");
37            conn.setRequestProperty("Connection", "keep-alive");
38            conn.setRequestProperty("Accept", "application/json");
39            conn.setRequestProperty("Origin", "http://localhost");
40            conn.setRequestProperty("Accept-Encoding", 
41                                    "gzip,deflate,sdch");
42            conn.setRequestProperty("Accept-Language", "es-ES,es;q=0.8");
43            conn.setRequestProperty("Accept-Charset", "UTF-8"); 
44            conn.setRequestProperty("User-Agent", "Java");
45
46            // If set, send the authentication token
47            if(token != null) {
48                conn.setRequestProperty("Authorization", 
49                                        "Bearer " + token);
50            }
51
52            if (data != null) {
53                conn.setRequestProperty("Content-Type", 
54                    "application/json; charset=UTF-8");
55                conn.setRequestProperty("Content-Length", 
56                    Integer.toString(data.length()));
57                conn.setDoOutput(true);
58                //Send request
59                DataOutputStream wr = 
60                    new DataOutputStream(conn.getOutputStream());
61                wr.write(data.getBytes());
62                wr.flush();
63                wr.close();
64            }
65
66            String charset = getCharset(
67                                conn.getHeaderField("Content-Type"));
68
69            if (charset != null) {
70                InputStream input = conn.getInputStream();
71                if ("gzip".equals(conn.getContentEncoding())) {
72                    input = new GZIPInputStream(input);
73                }
74
75                bufInput = new BufferedReader(
76                                    new InputStreamReader(input));
77
78                String line;
79                while((line = bufInput.readLine()) != null) {
80                    result.add(line);
81                }
82            }
83        } catch (IOException e) {
84        } finally {
85            if (bufInput != null) {
86                try {
87                    bufInput.close();
88                } catch (IOException e) { }
89            }
90        }
91
92        return result.toString();
93    }
94}

As you can see, we have the getCharset method explained before to get the charset encoding for the communication. The getResponse method will be used to send a request to a web service. It has 3 parameters: the url to connect, the data to send in the body of the request (or null if there’s no data), and the operation or method (GET, POST, PUT, DELETE). This response will be stored in a String that will be returned from this static method.

Also, some HTTP headers have been established so that the request is similar to what a web browser would send, like the origin domain (in this case localhost), the preferred language (Spanish), the possibility to compress data (gzip) in order to save bandwidth, a time out for the connection, or the data type used for communication (application/json).

There are also some other methods and attributes to deal with tokens for client authentication, so that we can store the token provided by the server in a static variable and send it back to the server in every request. But we are not going to use them for now.

You will be provided with this class in the Virtual Classroom, so that you can use it in the exercises to help you connect and get data from the web services more quickly.

4.6.3. JSON processing

Nowadays, most web services send and receive information in JSON format (XML is almost abandoned for this use). This format is native of JavaScript but most languages like Java have the necessary tools to process it.

The basic information about JSON and the available tools for each language can be found at http://www.json.org/. To process this information we can use the org.json API (also present in Android) or other options like Google’s GSON, but there are a lot of options. We will see here how to use GSON library.

4.6.3.1. Using GSON

Let’s see an example of how to use GSON library. First of all, we need to add that library to our Java project. You can download the latest version of the JAR file in the Maven repository, or the version that you will find in the Virtual Classroom. You must add the JAR file as a global or local library to your IntelliJ project, as you did with JavaFX in previous units.

Imagine that we receive this information from a web service in JSON format:

 1{
 2	"error":false,
 3	"person": {
 4		"name":"Peter",
 5		"age":30,
 6		"address":[
 7			{"city":"London","street":"Some street 24"},
 8			{"city":"New York","street":"Other street 12"}
 9		]
10	}
11}

GSON will try to automatically convert from JSON to a native Java object. So it will need a class that contains the same fields as the JSON response (same name). There’s no need to create any specific constructor. For the example above, we need a class called Address with two attributes called city (String) and street (String), and another class called Person with the attributes name (String), age (int) and address (of type Address).

Now we need an additional class that maps the initial JSON response format:

 1public class GetPersonResponse {
 2    boolean error;
 3    Person person;
 4    
 5    public boolean getError() {
 6        return error;
 7    }
 8    
 9    public Person getPerson() {
10        return person;
11    }
12}

If the field names are correctly set, it will map everything automatically:

 1public static void main(String[] args) {
 2
 3    String json = 
 4        ServiceUtils.getResponse("http://localhost/services/example", 
 5            null, "GET");
 6
 7    if(json != null) {
 8        Gson gson = new Gson();
 9        GetPersonResponse personResp = gson.fromJson(json, 
10            GetPersonResponse.class);
11        if(!personResp.getError()) {
12            System.out.println(personResp.getPerson().toString());
13            System.out.println(personResp.getPerson().getClass());
14        } else {
15            System.out.println("There was an error in the request");
16        }
17    }
18}

For this example, we would need to override the toString method in both classes Person and Address to show their data in an appropriate format. Note how we use the ServiceUtils class explained above to get a reponse, and then use GSON library to parse the response and store the corresponding data in the appropriate objects, according to GetPersonResponse class.

If a class property’s name doesn’t match the JSON field name, we can tell the GSON parser that it has to assign that field by using an annotation with that property:

1@SerializedName("error")
2boolean haserror; // In JSON it will be named "error"

You can learn more about GSON library in this GSON tutorial.

Exercise 9

Create a Java project called JsonParsing. Add the GSON library on it, and then implement a program (a console application, not a JavaFX one) that uses previous code (ServiceUtils, Person, Address and GetPersonResponse classes, apart from the main application) to connect to a server and retrieve a person information.

You can use the Node server provided to you in this session’s resources, and access the localhost/services/example URL to get the JSON data back. You can also edit the code of this server to change the URI or the port, if you want to.

4.6.4. Accessing web services from a different thread

Connecting to a web service and getting a response can be a costly operation, specially if the Internet connection is not the best and/or the server is overloaded. If we access a web service in the main thread, the application will be blocked (unresponsive) until we get the result.

The best way to deal with web services (or any other remote connection) is by using a separate thread to start the connection and then process the results when they’re received. If processing those results implies changing the view in a JavaFX application, we can use a Service or the Platform.runLater() method, like in this example that calls the sum service example shown in previous sections.

 1public class GetSumService extends Service<Integer> {
 2
 3    int n1, n2;
 4
 5    public GetSumService(int n1, int n2) {
 6        super();
 7        this.n1 = n1;
 8        this.n2 = n2;
 9    }
10
11    @Override
12    protected Task<Integer> createTask() {
13        return new Task<Integer>() {
14            @Override
15            protected Integer call() throws Exception {
16                BufferedReader bufInput = null;
17                Integer result = 0;
18                try {
19                    URL url = new URL("http://localhost/services/sum/" + 
20                        n1 + "/" + n2);
21                    URLConnection conn = url.openConnection();
22                    
23                    bufInput = new BufferedReader(
24                    new InputStreamReader(conn.getInputStream()));
25                    result = Integer.parseInt(bufInput.readLine()); 
26                } catch (IOException e) {} finally {
27                    if(bufInput != null) {
28                        try {
29                            bufInput.close();
30                        } catch (IOException e) {}
31                    }
32                }
33                
34                Thread.sleep(5000); // simulate a 5 seconds delay!
35                return result;
36            }
37        };
38    }
39    
40}

If this is the view…

In the application’s controller, when we click “Add” button, we’ll create and start the service, and update the corresponding label when it’s finished:

 1private void sumNumbers(ActionEvent event) {
 2    gss = new GetSumService(
 3                  Integer.parseInt(num1.getText()), 
 4                  Integer.parseInt(num2.getText()));
 5    gss.start();
 6    addButton.setDisable(true);
 7    resultLabel.setVisible(false);
 8        
 9    gss.setOnSucceeded(e -> {
10        resultLabel.setText("Result: " + gss.getValue());
11        addButton.setDisable(false);
12        resultLabel.setVisible(true);
13    });
14}

Exercise 10

Create a JavaFX project called FXWebServiceExample and create a JavaFX application similar to the one shown above. Use the GetSumService class to access the sum service and retrieve the sum of the two digits sent as parameters.

As in previous exercise, you can use the Node services provided to you in this session. In this case, you should access localhost/services/sum with the two parameters needed (for instance, localhost/services/sum/5/2 should return 7 as a result). You can also change the URI or port in the Node project, if you want to.