Implement Golang RPC Service Based on Apache Thrift

Datetime:2016-08-23 03:10:21          Topic: PHP  Golang           Share

Remote Procedure Call (RPC) is a protocol that one program can use to request a service from a program located in another computer in a network without having to understand network details. (A procedure call is also sometimes known as a function call or a subroutine call.) RPC uses the client/server model.

The Apache Thrift software framework, for scalable cross-language services development, combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml and Delphi and other languages.

Thrift Network Stack

  • Server - Single-threaded, event-driven etc.
  • Processor - Compiler generated.
  • Protocol - JSON, compact etc.
  • Transport - Raw TCP, HTTP etc.

Protocol Layer

The protocol layer provides serialization and deserialization. Thrift supports the following protocols:

  • TBinaryProtocol - A straight-forward binary format encoding numeric values as binary, rather than converting to text.
  • TCompactProtocol - Very efficient, dense encoding of data (See details below).
  • TDenseProtocol - Similar to TCompactProtocol but strips off the meta information from what is transmitted, and adds it back in at the receiver. TDenseProtocol is still experimental and not yet available in the Java implementation.
  • TJSONProtocol - Uses JSON for encoding of data.
  • TSimpleJSONProtocol - A write-only protocol using JSON. Suitable for parsing by scripting languages.
  • TDebugProtocol - Uses a human-readable text format to aid in debugging.

Tranport Layer

The transport layer is responsible for reading from and writing to the wire. Thrift supports the following:

  • TSocket - Uses blocking socket I/O for transport.
  • TFramedTransport - Sends data in frames, where each frame is preceded by a length. This transport is required when using a non-blocking server.
  • TFileTransport - This transport writes to a file. While this transport is not included with the Java implementation, it should be simple enough to implement.
  • TMemoryTransport - Uses memory for I/O. The Java implementation uses a simple ByteArrayOutputStream internally.
  • TZlibTransport - Performs compression using zlib. Used in conjunction with another transport. Not available in the Java implementation.

Install Thrift

Install Apache Thrift on OS X via Homebrew and validate Thrift version:

$ brew install thrift && thrift -version

Thrift IDL

The Thrift interface definition language (IDL) allows for the definition of Thrift Types. A Thrift IDL file is processed by the Thrift code generator to produce code for the various target languages to support the defined structs and services in the IDL file. The first thing to know about are types. The available types in Thrift are:

  • bool - Boolean, one byte
  • i8 (byte) - Signed 8-bit integer
  • i16 - Signed 16-bit integer
  • i32 - Signed 32-bit integer
  • i64 - Signed 64-bit integer
  • double - 64-bit floating point value
  • string - String
  • binary - Blob (byte array)
  • map<t1,t2> - Map from one type to another
  • list<t1> - Ordered list of one type
  • set<t1> - Set of unique elements of one type

Create mythrift.thrift file:

/**
 * Thrift files can namespace, package, or prefix their output in various
 * target languages.
 */

namespace go mythrift.demo
namespace php mythrift.demo

/**
 * Structs are the basic complex data structures. They are comprised of fields
 * which each have an integer identifier, a type, a symbolic name, and an
 * optional default value.
 *
 * Fields can be declared "optional", which ensures they will not be included
 * in the serialized output if they aren't set.  Note that this requires some
 * manual management in some languages.
 */
struct Article{
    1: i32 id,
    2: string title,
    3: string content,
    4: string author,
}

const map<string,string> MAPCONSTANT = {'hello':'world', 'goodnight':'moon'}

service myThrift {
        list<string> CallBack(1:i64 callTime, 2:string name, 3:map<string, string> paramMap),
        void put(1: Article newArticle),
}

Compile IDL File:

$ thrift -r --gen go mythrift.thrift
$ thrift -r --gen php mythrift.thrift
$ thrift -r --gen php:server mythrift.thrift

After execute above command will generate a gen-go folder, and copy gen-go/mythrift folder to $GOPATH .

Get Golang Thrift package:

$ go get github.com/apache/thrift/lib/go/thrift

Golang Implement Thrift Server

Create main.go file in thrift-server folder:

package main

import (
    "fmt"
    "os"
    "time"

    "git.apache.org/thrift.git/lib/go/thrift"
    "mythrift/demo" // Import interface package generated by Thrift
)

const (
    NetworkAddr = "127.0.0.1:9090"
)

type mythriftThrift struct{}

func (this *mythriftThrift) CallBack(callTime int64, name string, paramMap map[string]string) (r []string, err error) {
    fmt.Println("-->from client Call:", time.Unix(callTime, 0).Format("2006-01-02 15:04:05"), name, paramMap)
    r = append(r, "key:"+paramMap["a"]+"    value:"+paramMap["b"])
    return
}

func (this *mythriftThrift) Put(s *demo.Article) (err error) {
    fmt.Printf("Article--->id: %d\tTitle:%s\tContent:%s\tAuthor:%s\n", s.ID, s.Title, s.Content, s.Author)
    return nil
}

func main() {
    transportFactory := thrift.NewTFramedTransportFactory(thrift.NewTTransportFactory())
    protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()
    serverTransport, err := thrift.NewTServerSocket(NetworkAddr)
    if err != nil {
        fmt.Println("Error!", err)
        os.Exit(1)
    }

    handler := &mythriftThrift{}
    processor := demo.NewMyThriftProcessor(handler)

    server := thrift.NewTSimpleServer4(processor, serverTransport, transportFactory, protocolFactory)
    fmt.Println("thrift server in", NetworkAddr)
    server.Serve()
}

Golang Implement Thrift Client

Create main.go file in thrift-client folder:

package main

import (
    "fmt"
    "net"
    "os"
    "strconv"
    "time"

    "git.apache.org/thrift.git/lib/go/thrift"
    "mythrift/demo"
)

const (
    HOST = "127.0.0.1"
    PORT = "9090"
)

func main() {
    startTime := currentTimeMillis()

    transportFactory := thrift.NewTFramedTransportFactory(thrift.NewTTransportFactory())
    protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()

    transport, err := thrift.NewTSocket(net.JoinHostPort(HOST, PORT))
    if err != nil {
        fmt.Fprintln(os.Stderr, "error resolving address:", err)
        os.Exit(1)
    }

    useTransport := transportFactory.GetTransport(transport)
    client := demo.NewMyThriftClientFactory(useTransport, protocolFactory)
    if err := transport.Open(); err != nil {
        fmt.Fprintln(os.Stderr, "Error opening socket to "+HOST+":"+PORT, " ", err)
        os.Exit(1)
    }
    defer transport.Close()

    for i := 0; i < 10; i++ {
        paramMap := make(map[string]string)
        paramMap["a"] = "mythrift.demo"
        paramMap["b"] = "test" + strconv.Itoa(i+1)
        r1, _ := client.CallBack(time.Now().Unix(), "Go client", paramMap)
        fmt.Println("Go client call->", r1)
    }

    model := demo.Article{1, "Send from Golang Thrift Client", "Hello World!", "Golang"}
    client.Put(&model)
    endTime := currentTimeMillis()
    fmt.Printf("The call took:%d-%d=%d Millis \n", endTime, startTime, (endTime - startTime))

}

func currentTimeMillis() int64 {
    return time.Now().UnixNano() / 1000000
}

PHP Implement Thrift Client

Clone [email protected]

:apache/thrift.gitand import thrift/lib/php/lib in PHP script, create phpthrift-client folder copy gen-php folder into it. Create main.php in phpthrift-client

folder:

<?php

    /**
     * Thrift RPC - PHP Client
     */
    namespace mythrift\demo;
    error_reporting(E_ALL);
    $startTime  = getMillisecond();
    $ROOT_DIR   = realpath(dirname(__FILE__).'/');
    $GEN_DIR    = realpath(dirname(__FILE__).'/') . '/gen-php';

    require_once $ROOT_DIR . '/Thrift/ClassLoader/ThriftClassLoader.php';

    use Thrift\ClassLoader\ThriftClassLoader;
    use Thrift\Protocol\TBinaryProtocol;
    use Thrift\Transport\TSocket;
    use Thrift\Transport\TSocketPool;
    use Thrift\Transport\TFramedTransport;
    use Thrift\Transport\TBufferedTransport;

    $loader     = new ThriftClassLoader();
    $loader->registerNamespace('Thrift',$ROOT_DIR);
    $loader->registerDefinition('mythrift\demo', $GEN_DIR);
    $loader->register();

    $thriftHost = '127.0.0.1';
    $thriftPort = 9090;

    $socket     = new TSocket($thriftHost,$thriftPort);
    $socket->setSendTimeout(10000); // Sets the send timeout.
    $socket->setRecvTimeout(20000); // Sets the receive timeout.
    $transport  = new TFramedTransport($socket);
    $protocol   = new TBinaryProtocol($transport);
    $client     = new \mythrift\demo\myThriftClient($protocol);

    $transport->open();
    $socket->setDebug(TRUE);

    for ($i=0; $i < 10; $i++) {
         $item      = array();
         $item['a'] = 'mythrift.demo';
         $item['b'] = 'test' + $i;
         $result    = $client->CallBack(time(), 'PHP Client', $item);
         echo 'PHP Client Call->', implode('', $result), PHP_EOL;
    }

    $s             = new \mythrift\demo\Article();
    $s->id      = 1;
    $s->title   = 'Send from PHP Thrift Client';
    $s->content = 'Hello World!';
    $s->author  = 'PHP';
    $client->put($s);

    $endTime       = getMillisecond();

    echo 'The call took:', $endTime, '-', $startTime, '=', ($endTime - $startTime), ' Millis', PHP_EOL;

    function getMillisecond() {
        list($t1, $t2) = explode(' ', microtime());
        return (float)sprintf('%.0f', (floatval($t1) + floatval($t2)) * 1000);
    }

    $transport->close();

Build and Testing

Build and run thrift-server :

$ go build thrift-server && ./thrift-server
thrift server in 127.0.0.1:9090

Build and run thrift-client :

$ go build thrift-client && ./thrift-client
Go client call-> [key:mythrift.demo    value:test1]
Go client call-> [key:mythrift.demo    value:test2]
Go client call-> [key:mythrift.demo    value:test3]
Go client call-> [key:mythrift.demo    value:test4]
Go client call-> [key:mythrift.demo    value:test5]
Go client call-> [key:mythrift.demo    value:test6]
Go client call-> [key:mythrift.demo    value:test7]
Go client call-> [key:mythrift.demo    value:test8]
Go client call-> [key:mythrift.demo    value:test9]
Go client call-> [key:mythrift.demo    value:test10]
The call took:1467012021504-1467012021428=76 Millis

Thirft server output:

-->from client Call: 2016-06-27 15:20:21 Go client map[a:mythrift.demo b:test1]
-->from client Call: 2016-06-27 15:20:21 Go client map[b:test2 a:mythrift.demo]
-->from client Call: 2016-06-27 15:20:21 Go client map[b:test3 a:mythrift.demo]
-->from client Call: 2016-06-27 15:20:21 Go client map[a:mythrift.demo b:test4]
-->from client Call: 2016-06-27 15:20:21 Go client map[b:test5 a:mythrift.demo]
-->from client Call: 2016-06-27 15:20:21 Go client map[a:mythrift.demo b:test6]
-->from client Call: 2016-06-27 15:20:21 Go client map[b:test7 a:mythrift.demo]
-->from client Call: 2016-06-27 15:20:21 Go client map[b:test8 a:mythrift.demo]
-->from client Call: 2016-06-27 15:20:21 Go client map[a:mythrift.demo b:test9]
-->from client Call: 2016-06-27 15:20:21 Go client map[a:mythrift.demo b:test10]
Article--->id: 1    Title:Send from Golang Thrift Client   Content:Hello World!    Author:Golang

Run phpthrift-client/main.php :

$ php main.php
PHP Client Call->key:mythrift.demo    value:0
PHP Client Call->key:mythrift.demo    value:1
PHP Client Call->key:mythrift.demo    value:2
PHP Client Call->key:mythrift.demo    value:3
PHP Client Call->key:mythrift.demo    value:4
PHP Client Call->key:mythrift.demo    value:5
PHP Client Call->key:mythrift.demo    value:6
PHP Client Call->key:mythrift.demo    value:7
PHP Client Call->key:mythrift.demo    value:8
PHP Client Call->key:mythrift.demo    value:9
The call took:1467018064708-1467018064646=62 Millis

Thirft server output:

-->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:0]
-->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:1]
-->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:2]
-->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:3]
-->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:4]
-->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:5]
-->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:6]
-->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:7]
-->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:8]
-->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:9]
Article--->id: 1    Title:Send from PHP Thrift Client   Content:Hello World!    Author:PHP

Implement Golang RPC Service Based on Apache Thrift

  • 4.67 / 5 5
  • 1 / 5
  • 2 / 5
  • 3 / 5
  • 4 / 5
  • 5 / 5

3votes, 4.67 avg. rating ( 93 % score)





About List