Network Programming

Written by Daniel Campos.
Some fixes by Benoît Minisini./

Introduction

The future, at least the forseeable future, is going to be based on networks and standards. Even large companies that in the past tried to hide their internal file formats, for the most part, are begining to see the light, espousing the virtues of XML, SOAP, RPC, and other tools which, if they had put some effort and money into them in the past would probably be even more useful to them today.

Think that...

Today, devices with no network connection are difficult to imagine. People want to know what is happening even if it is on the other side of the globe. Customers expect devices made by different manufacturers to be compatible with each other, and see the problem of making them compatible as the pervue of the device makers. So business need well-documented standards in order to share information, they need to control their money, information is power and presents opportunity to grow and earn. Big hardware and software companies are trusted to make this happen.

So...

New back-office systems provided by main-stream companies are based on known standards, mainly : data bases (SQL), http servers (like Apache), and other standardized ways of storing data and communicating such as XML, RPC and SOAP. If big companies use these tools, medium and small companies will have to do the same...or risk not being able to do business with their larger counterparts.

XML, RPC and SOAP

Someone learning XML may, when starting out, think "what a stupid thing! XML appears to consist of tags and random data in a text file", and that is indeed what it is. The power of XML is subtle, it lies in being an international standard that can easily be read by both people and computers. A student of XML doesn't have to be a programmer to understand it and put it to good use. Their knowledge, a computer, and a text editor is all that is required. To aid us humans there are many software tools available to allow us to share our XML documents, or translate them to html or even beautiful PDF reports. HTTP (Hyper Text Transport Protocol) is, as its name implies, a protocol, and a protocol is to data communication what languages are to people. Like languages, there are many protocols, and like languages if two parties have one in common they can communicate and share information. The birth of the Arpanet and the growth of the Internet made HTTP popular; it is used to send and receive web pages, images, commercial data, and files. In short pretty much anything you can imagine receiving over the Internet can be sent using HTTP from any server anywhere in the world. Almost all network hardware (routers, firewalls, servers, and end-user computers) use HTTP. Using HTTP to send and receive information means not worrying about the physical location of the destination; all of the network hardware in the world will carry that information without asking questions, we just press the right button (well, usually the left button of the mouse), and let the big machine that is the Internet, do all of the work. The network (Internet or Intranet) is where we also find the XML-RPC and SOAP protocols.

XML-RPC and SOAP allow computers to talk amongst themselves. Using these protocols they can do more than share some documents, one computer can send orders to another computer to perform a task and the second computer will return the result of the action. You can find this on your own personal computer, some programs use libraries and other programs to perform work. The only new thing in XML-RPC and SOAP, is that now the conversations are not only within your PC, but can take place between computers on a network. This is a big deal. While the idea of interprocess and interprogram communication is not new, it is now not constrained to a single computer but can operate across network and data format standards. HTTP is used to carry the information, XML is used to write orders, questions, and answers. The end result of this is that today it is quite trivial for programmers (and even non-programmers), to write programs that will be connected with other programs running on other computers. Even if they are in different parts of the world! Oh, and one more really important point, you don't need to have a specific hardware-software platform to do any of this. These are standards, they work on hardware and software from many different providers. What are your wants/needs? Low price? High performance? Scalability? Shop, compare, and buy whatever fits your criteria.

Gambas must be there

Gambas needs to implement standards, and I'd say more : Linux needs Gambas. Gambas is a BASIC language interpreter. BASIC is an easy language to learn, in fact the name of the language is an acronym derived from Beginners All-purpose Symbolic Instruction Code. Anybody can learn BASIC and start making little programs and scripts. A lot of programmers started learning BASIC when they were children, maybe on 8-bit CPUs. A lot of administrative people in business and the government know at least a little of the BASIC language, they are not programmers, but they have what they need and can write BASIC macros to help them work with small and medium sized data bases, spread-sheets and text documents. They know how to create a formula, write and read data from tables, and share information by sending it via e-mail. On the other side of this coin there are legions of web developers who are too busy to learn about the intricacies of pointers and memory management of languages like 'C'. They have to write html, create beautiful pictures, administrate servers, deal with hackers and many other things. They need a simple language to create CGI's. There are languages like PHP or Perl, and there's one called ASP (which is not much more than BASIC mixed with HTML), so even if they are unaware of it, they already know BASIC.

The conclusion drawn from the above is that there are many many people waiting (whether they know it or not) to migrate to BASICLinux. What can we do here? We are in the middle of the road, GAMBAS has a simple and well-designed GUI (not very many projects can say that), we have the interpreter with lots of libraries written in C or C++ waiting to help us, and perhaps the biggest API that any operating system has ever known. To cross to the other side we need only to implement those standards that will allow programmers and users to connect to the information they need. Network component is a major stepping stone in accomplishing this goal. Let's open the door and invite the world in.

Network component

It is not really one component, it will be implemented in (at least) three components: gb.net, gb.net.curl and gb.xml.

gb.net is the foundation:

Currently the web is based on TCP/IP protocols. We can say that the IP part of this protocol is matter for each individual operating system, what we have to think about is the TCP portion. To implement everything we have the following classes:

• DnsClient: is a class to convert host names (like "Gambas.sourceforge.net"), which are understood by humans to IP addresses (like 192.168.0.1) which are the real names for computers. It can perform its work using operating systems resources: DNS, NIS, hosts file,NMB lookup, LDAP, etc...

• Socket : to share information with servers, we need to stablish a TCP connection. This class can do that, and send and receive data in "raw" format.

• ServerSocket: sometimes we'll be the servers, and we need to let clients connect with us. This is the class to do it using TCP/IP.

• UdpSocket: some internet tasks, like transfering multimedia streaming data, do not work fine if they use a connection, so they usually use UDP, a more "primitive" way of transfering data, with no flow control. Datagram class is intended to create both UDP clients and servers.

• Local or Unix sockets: sometimes we don't need to connect with other computers, but with other programs in the same computer. Using TCP for this is not efficient, and you waste a limited resource as TCP ports are. Unix sockets are local sockets implemented by the operating system to "emulate" TCP sockets into the computer ( oh, well, this is not a technical conference, I simply try to give an idea, you know? ), and data sharing using this method is quite fast and inexpensive. Both ClientSocket? and ServerSocket? can also establish unix-socket connections.

• SerialPort: now let's speak about the rest of computing world. Not all devices, specially designed for industrial or commerce tasks have an ethernet connection. May be 70% of them have RS-232, RS-485, or RS-422 serial ports. And programmers often have to stract or send information to them, and then compute the information and convert it to data to be placed in a database, web pages, and so on. So, there's also a class called "SerialPort" to allow send and receive data using a serial port.

SerialPort, Socket and UdpSocket classes inherits from generic Stream class. This means you can use them as file descriptors,using standard Gambas methods like "read", "write", or "close", so you will find easy to learn using them.

gb.net.curl

'libcurl' is a free and portable library written by some people at http://curl.haxx.se. It provides all the pieces needed to manage high level network protocols, like HTTP, FTP and TELNET. This library will be useful to write a component with the following classes:

• HttpClient : by using ClientSocket? and ServerSocket? you'll receive only raw data, that you have to manage and verify. This client allows you to connect with HTTP servers (like Apache, yes), managing all HTTP protocol, you have just to care about the document (or object, as DOM people like to say) you receive. (this class is in ALPHA state)

• FtpClient : FTP protocol is one of the basic tools for any company: they can send and receive data, for example in ASCII format, and then proccess it to consolidate data in a database. (not yet implemented)

• TelnetClient? : Allowing Gambas programs to remote control other machines. (not yet implemented)

gb.xml are the tools to connect with the world

Note that gb.xml is not yet fully implemented.

• There will be at least two different pure-XML classes:
  • A tree based class : it saves all XML information in memory, which makes work with XML very easy, you can move to any part of the document, and modify it easily, and using very few time.

  • An XMLReader class, more conventional : it just reads pieces of XML document from a file, without keeping it in memory. May be it is more slow, but when you have a big document, a tree based model can use all your memory resources, even hanging the computer.

  • May be a SAX based XML class : it is useful to parse documents while you are receiving it from a stream. (it is useful for other things, too)

• There will be XML-RPC client class : it will allow you to call remote RPC-servers and receive data just as if you were calling methods from your own program, or at least in a quite similar way. Nothing about TCP/IP or XML, just Integers, Strings, Arrays...

• There must be also a XML-RPC server, for two reasons : the first, to "decrypt" incoming information from client in XML format to data types, and second, to allow create stand-alone servers without the need of "monsters" like Apache.

In the future there will be work on SOAP front. The idea of SOAP is quite similar to XML-RPC, it is also based on XML and HTTP, but with SOAP you can perform very complex tasks, it is powerful, but by the other side is quite complex.

Programming with the gb.net component

Connected Sockets: Clients and Servers

A socket is just a place from where you can read and write data. However, socket definition does not speak about servers, clients, connections, etc, this is a concept placed in the next floor. TCP and Local sockets are particular socket implementations in which there are flow control and specifications about the program roles: client and server.

There are two ways to work with connected sockets:

• TCP sockets: you can use them to communicate with remote or local programs.

• Local or Unix sockets: just to communicate with programs in your own machine.

Now let's implement two programs to play with TCP and Local sockets:

• The client will connect with server and it will send an string 'hello' to the server.

• The server will return 'bye'.

• The client will close the connection after bye message reading.

The Client Side : Acting like a TCP or Local (Unix) Client

To create a program in which you will connect to a remote or local server using TCP sockets or Local sockets, you have to use "Socket" class. We will use here two ways to implement that, the first implementation does not use any event:

1. Create a new project called "Sock1".

2. Go to "project","properties","components", remove "qt" and add "net", then press "OK"

3. Add a new module called M1.

4. Edit the following code...

Before the Main() method, let's define an object from 'Socket' class:

Now, the first thing we do is initialize the object: Then, we have to connect with remote or local server, so we need to use Connect() method. If we want to connect using TCP protocol, we need to specify remote host name or ip, and port to connect to: If we need to connect to a Local server (UNIX sockets), we have to specify path from that socket: The connection process takes a time, so we can not start sending or receiving data in the next line of code, instead, we have to wait until connection is finished or there's an error trying to connect. If we really connect, "Status" property from Socket will change to value Socket.Connected (value 7), but if connection process fails, Status property will change to a value less than zero. So, we'll wait in a loop... Note that you must use "Wait" method, to let the event loop update Socket Status. After that, let's indicate an error code, something failed, or, if connection was stablished, let's send some data: Now, let's read reply from server. First, we wait until there's data to read, checking it with Lof method. Then, we will read data: Finally, we close the socket: That's all.

The full code is:

Now, let's implement another version, by using events and not active waiting.

1. Create a new project called "Sock2".

2. Go to "project","properties","components", remove "qt" and add "net", then press "OK"

3. Add a new class called "ClsMain".

4. Edit that code:

We need an instance of our class, and a Socket:

At _New method from our class, we'll start socket connection: At main method, we create a new instance of our class, so we start connection, as _New method is called when creating an object : When connection has been established successfully, "Ready" event from socket raises, and we send our string to server : But, if there was an error, the Error event raises : When new data arrives, the Read event raises, so we read that data, put it on screen, and close the socket :

Here is the full code:

The Server Side : Acting like a TCP or Local (Unix) Server

To work a a server, we need 'ServerSocket' class. It listen for connections, and returns a new Socket object for each client connection, so we can manage multiple clients.

1. Create a new project called "Srv".

2. Go to "project","properties","components", remove "qt" and add "net", then press "OK"

3. Add a new class called "ClsServer".

4. Edit that code:

As we did in the second client example, we need an instance of our class. We also need a ServerSocket? object, and an array of objects to place the Socket objects that we need to communicate with our clients.

At program start, we create the instance of our class. When class is loaded by the interpreter, it calls to _New method. We initialize here the array of objects, and start the server. To do that, we have to specify TCP port to listen to, type of socket, and call to 'Listen' If we'd like to use Local or Unix sockets, instead of TCP sockets, we should specify Local socket type, and a path instead of a port When server is listening, each time a client tries to connect to our service, 'Connection' event from server raises. Here we must accept that connection by calling 'Accept' method. It returns to us a 'Socket' object to manage that connection, so there will be a 'Socket' object for each client connection, that we store in our object array Our client will send us 'hello' message, and then 'Read' event from 'Socket' will raise. Note that we use here 'LAST' keyword to know which of our clients has sent that message. Here we read that string, and send our string 'bye' Finally, after client has closed the conection, we receive 'Closed' event, and clear that 'Socket' object from our array, as that connection is alive no more The full code is: