Pagination with Hbase

Pagination with Hbase

Pagination with Hbase

Pagefilter is use for pagination and it controls how many row per page should be returned.

Filter filter = new PageFilter(15);

Hbase pagination always work in forward direction, there are no way to go reverse or in backward direction.

suppose you need to implement functionality at UI pagination table and it should have next/forward and back/backward feature.

Forward option easily implemented using PageFilter but for backward, no solution. e.g. you need to get the rows(20) from end key to most recent and going backward. eg. backward get all records 1000 – 980.
Unless your rows are predictable (you know how many rows between row key 0 and row key 2^32) reverse scanning won’t even matter but If the rows are unpredictable, you would always have to scan the entire set to find out how many pages there and get the particular range start and stop row key.

I worked on similar use case, alternate solutions can be if your dataset is very huge-

  1. Secondary index table-
  2. each insert into main table, create a entry into secondary table that’s sorted in reverse order. whenever request comes for scan in backward, scan the reverse/ index table for number of rows then use the returned start and stop row key to query on actual table.

  3. Pagination table-
  4. create a new hbase table with one cf family.

    create 'PAGINATION', {NAME => 'CF'}

    Create java bean-

    1
    2
    3
    4
    5
    6
    
    public class Pagination {
        protected int numberOfRecords;
        protected String startKey;
        protected int pageNumber;
        protected String scrollingType;
    }

    scrolling type is forward/backward.

    for each forward search, create an entry into pagination table with row key ‘search criteria’+numberOfRecords from pagination bean, column qualifier ‘CF:pageNumber’ and value is start key of set.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    
     public void put( String key, Pagination pagination )
          throws DAOException
        {
          HTableInterface table = null;
          try {
            table = getTable(  "PAGINATION"  );
            Put put = new Put( Bytes.toBytes( key + "_" + pagination.getNumberOfRecords() ) );
            put.add( Bytes.toBytes( "CF"), Bytes.toBytes( pagination.getPrevPageNumber() ), Bytes.toBytes( pagination.getStartKey() ) );
            put.add( Bytes.toBytes( "CF"), Bytes.toBytes( "LAST_ACCESSED" ), Bytes.toBytes( System.currentTimeMillis() ) );
            table.put( put );
          } catch( IOException e ) {
            StringWriter sw = new StringWriter();
            e.printStackTrace( new PrintWriter( sw ) );
            logger.error( sw.toString() );
            throw new DAOException( "Failed to update: " + e.getMessage() );
          } finally {
            closeResources( table );
          }
          logger.info( "WSPagination.put: end" );
        }

    if required, maintain the last accessed extra column for clean up job that clean up the table periodically.
    whenever request comes for scan in backward, get the start key from pagination table using ‘search criteria’+numberOfRecords and pageNumber, then search on actual table. each requests should contains pagination bean.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    
     public String get( String key, Pagination pagination )
          throws DAOException
        {
          HTableInterface table = null;
          String resultStr = null;
          try {
            table = getTable(  "PAGINATION"  );
            logger.debug( "get key:" + key + DatabaseConstants.FIELD_DELIMITER_TRANS + pagination.getNumberOfRecords() + "page: "
              + ( pagination.getPrevPageNumber() ) );
            Get get = new Get( Bytes.toBytes( key + "_"+ pagination.getNumberOfRecords() ) );
            get.addColumn( Bytes.toBytes( "CF"), Bytes.toBytes( pagination.getPrevPageNumber() ) );
            Result result = table.get( get );
            resultStr = ( result == null || result.isEmpty() ) ? null : Bytes.toString( result.getValue(
              Bytes.toBytes( WS_PAGINATION.FAMILY ), Bytes.toBytes( pagination.getPrevPageNumber() ) ) );
          } catch( Exception e ) {
            StringWriter sw = new StringWriter();
            e.printStackTrace( new PrintWriter( sw ) );
            throw new DAOException( "Failed to get Number: " + resultStr + "\t error: " + e.getMessage() );
          } finally {
            closeResources( table );
          }
          return resultStr;
        }

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.