JDBC River parameters

Jörg Prante edited this page on 23 Jan 2014 · 3 revisions

Pages 15

• Home
• Bulk indexing
• How bulk indexing is used by the JDBC river
• JDBC plugin feeder mode as an alternative to the deprecated Elasticsearch River API
• JDBC River parameters
• Labeled columns
• Moving a table into Elasticsearch
• Oracle schedule example script
• Quickstart
• RiverSource, RiverMouth, and RiverFlow
• Step by step recipe for setting up the river with PostgreSQL
• Step by step recipe for setting up the river with SQL Server
• Strategies
• Structured Objects
• Using a series of SQL statements

Clone this wiki locally

Clone in Desktop

Overview of the default parameter settings:

{
    "jdbc" :{
        "strategy" : "simple",
        "url" : null,
        "user" : null,
        "password" : null,
        "sql" : null,
        "schedule" : null,
        "poolsize" : 1,
        "rounding" : null,
        "scale" : 2,
        "autocommit" : false,
        "fetchsize" : 10, /* Integer.MIN for MySQL */
        "max_rows" : 0,
        "max_retries" : 3,
        "max_retries_wait" : "30s",
        "locale" : Locale.getDefault().toLanguageTag(),
        "index" : "jdbc",
        "type" : "jdbc",
        "bulk_size" : 100,
        "max_bulk_requests" : 30,
        "bulk_flush_interval" : "5s",
        "index_settings" : null,
        "type_mapping" : null
    }
}

strategy -the strategy of the JDBC river, currently implemented: "simple", "column"

url - the JDBC URL

user - the JDBC database user

password - the JDBC database password

sql - SQL statement(s), either a string or a list. If a statement ends with .sql, the statement is looked up in the file system. Example for a list of SQL statements:

"sql" : [
    {
        "statement" : "select ... from ... where a = ?, b = ?, c = ?",
        "parameter" : [ "value for a", "value for b", "value for c" ],
        "callable" : false
    },
    {
        "statement" : ...
    }
]

parameter - bind parameters for the SQL statement (in order)

callable - boolean flag, if true, the SQL statement is interpreted as a JDBC CallableStatement (default: false)

schedule - a cron expression for scheduled execution. Syntax is equivalent to the Quartz cron expression format and is documented at http://jprante.github.io/elasticsearch-river-jdbc/apidocs/org/xbib/elasticsearch/river/jdbc/support/cron/CronExpression.html

poolsize - the pool size of the thread pool that executes the scheduled SQL statements

rounding - rounding mode for numeric values: "ceiling", "down", "floor", "halfdown", "halfeven", "halfup", "unnecessary", "up"

scale - the precision of the numeric values

autocommit - true if each statement should be automatically executed

fetchsize - the fetchsize for large result sets, most drivers implement fetchsize to control the amount of rows in the buffer while iterating through the result set

max_rows - limit the number of rows fetches by a statement, the rest of the rows is ignored

max_retries - the number of retries to (re)connect to a database

max_retries_wait - the time that should be waited between retries

locale - the default locale (used for parsing numerical values, floating point character)

index - the Elasticsearch index used for indexing the data from JDBC

type - the Elasticsearch type of the index used for indexing the data from JDBC

bulk_size - the length of each bulk index request submitted

max_bulk_requests - the maximum number of concurrent bulk requests

bulk_flush_interval - the time period the bulk processor is flushing outstanding documents

index_settings - optional settings for the Elasticsearch index

type_mapping - optional mapping for the Elasticsearch index type