Package ghidra.features.bsim.query.client

Class AbstractSQLFunctionDatabase<VF extends LSHVectorFactory>

java.lang.Object
ghidra.features.bsim.query.client.AbstractSQLFunctionDatabase<VF>
Type Parameters:
VF - vector factory implementation class
All Implemented Interfaces:
FunctionDatabase, SQLFunctionDatabase, AutoCloseable
Direct Known Subclasses:
H2FileFunctionDatabase, PostgresFunctionDatabase
public abstract class AbstractSQLFunctionDatabase<VF extends LSHVectorFactory> extends Object implements SQLFunctionDatabase
Defines the BSim FunctionDatabase backed by an SQL database. Simple, one-column tables that only contain string data use the SQLStringTable class and are defined in this class. More complex tables are defined in their own classes in the ghidra.features.bsim.query.client.tables package.

  • Field Details

  • Constructor Details

    • AbstractSQLFunctionDatabase

      protected AbstractSQLFunctionDatabase(BSimJDBCDataSource ds, VF vectorFactory, int supportedLayoutVersion)

  • Method Details

    • close

      public void close()
      Description copied from interface: FunctionDatabase
      Close down (the connection with) the database
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface FunctionDatabase

    • closing

      protected void closing(Connection c)
      Perform any required cleanup prior to connection close. Partial cleanup has already been performed.
      Parameters:
      c - database connection

    • initConnection

      protected Connection initConnection() throws SQLException
      Establish PostgreSQL DB pooled Connection for this instance.
      Returns:
      database connection
      Throws:
      SQLException - if error occurs obtaining connection

    • beginTransaction

      protected Connection beginTransaction(boolean lockTablesForWrite) throws SQLException
      Start a transaction which will not be auto-committed. The endTransaction(boolean) method must be invoked when the transaction has completed or failed and should be performed within a finally block.
      Parameters:
      lockTablesForWrite - if true a table lock will be placed which will clear when endTransaction(boolean) is invoked.
      Returns:
      the connection associated with the transaction
      Throws:
      SQLException - if an error occurs

    • lockTablesForWrite

      protected void lockTablesForWrite() throws SQLException
      Lock appropriate tables during complex write/update operation
      Throws:
      SQLException - if an error occurs

    • endTransaction

      protected void endTransaction(boolean commit) throws SQLException
      End a transaction which was started with beginTransaction(boolean) and return DB connection to an auto-commit state.
      Parameters:
      commit - true if transaction should be commited, false to force a rollback.
      Throws:
      SQLException - if an error occurs

    • initializeDatabase

      protected void initializeDatabase(Configuration config) throws SQLException
      Initialize table state and config from existing database. NOTE: it is left to specific implementation to override this method to properly initialize default config.weightfactory and config.idflookup .
      Parameters:
      config - the database configuration
      Throws:
      SQLException - if there is a problem opening the database connection

    • generateRawDatabase

      protected abstract void generateRawDatabase() throws SQLException
      Generate new empty database which corresponds to ds. A new connection may be used to the default database so that a new database may be created.
      Throws:
      SQLException - if an error occurs while creating the database

    • createDatabase

      protected void createDatabase(Configuration config) throws SQLException
      Generate tables and initialize state for a new database.
      Parameters:
      config - the database configuration
      Throws:
      SQLException - if there is aproblem opening a connectino to the database

    • dropDatabase

      protected abstract void dropDatabase() throws SQLException
      Drop this database
      Throws:
      SQLException - if a database error occured

    • setConnectionOnTables

      protected void setConnectionOnTables(Connection db)

    • appendEscapedLiteral

      public static void appendEscapedLiteral(StringBuilder buf, String str) throws SQLException
      Parameters:
      buf - the string builder object
      str - the string to parse
      Throws:
      SQLException - if there is a zero byte in the string

    • convertDescriptionRows

      protected void convertDescriptionRows(SimilarityResult simres, List<DescriptionTable.DescriptionRow> descvec, VectorResult vecres, DescriptionManager res, SignatureRecord srec) throws SQLException, LSHException
      Convert a low-level list of function rows into full FunctionDescription objects
      Parameters:
      simres - (optional -- may be null) generate a SimilarityNote for every function
      descvec - is the list of low-level function rows
      vecres - (optional -- may be null) vector result producing these functions
      res - is the DescriptionManager holding the newly generated FunctionDescriptions
      srec - (optional -- may be null) is a description object of the vector
      Throws:
      SQLException - is there is an error querying tables
      LSHException - for internal consistency errors

    • deleteVectors

      protected abstract int deleteVectors(long id, int countdiff) throws SQLException
      Low level count decrement of a vector record from vectable, if count reaches zero, the record is deleted
      Parameters:
      id - vector row ID
      countdiff - the amount to subtract from count
      Returns:
      0 if decrement short of 0, return 1 if record was removed, return -1 if there was a problem
      Throws:
      SQLException - if there is a problem creating or executing the query

    • queryVectorId

      protected abstract VectorResult queryVectorId(long id) throws SQLException
      Parameters:
      id - the vector id
      Returns:
      the vector result
      Throws:
      SQLException - if there is a problem creating or executing the query

    • getStatus

      public FunctionDatabase.Status getStatus()
      Specified by:
      getStatus in interface FunctionDatabase
      Returns:
      the status of the current connection with this database

    • getConnectionType

      public FunctionDatabase.ConnectionType getConnectionType()
      Specified by:
      getConnectionType in interface FunctionDatabase
      Returns:
      the type of connection

    • getUserName

      public String getUserName()
      Specified by:
      getUserName in interface FunctionDatabase
      Returns:
      username (being used to establish connection)

    • getLSHVectorFactory

      public LSHVectorFactory getLSHVectorFactory()
      Specified by:
      getLSHVectorFactory in interface FunctionDatabase
      Returns:
      factory the database is using to create LSHVector objects

    • getInfo

      public DatabaseInformation getInfo()
      Specified by:
      getInfo in interface FunctionDatabase
      Returns:
      an information object giving general characteristics and descriptions of this database

    • compareLayout

      public int compareLayout()
      Description copied from interface: FunctionDatabase
      Return -1 if info layout version is earlier than current client expectation Return 1 if info layout version is later than current client expectation Return 0 if info version and client version are the same
      Specified by:
      compareLayout in interface FunctionDatabase
      Returns:
      comparison of actual database layout with layout expected by client

    • getServerInfo

      public BSimServerInfo getServerInfo()
      Description copied from interface: FunctionDatabase
      Return the server info object for this database
      Specified by:
      getServerInfo in interface FunctionDatabase
      Returns:
      the server info object

    • getURLString

      public String getURLString()
      Specified by:
      getURLString in interface FunctionDatabase

    • getLastError

      public FunctionDatabase.BSimError getLastError()
      Description copied from interface: FunctionDatabase
      If the last query failed to produce a response, use this method to recover the error message
      Specified by:
      getLastError in interface FunctionDatabase
      Returns:
      a String describing the error

    • initialize

      public boolean initialize()
      Description copied from interface: FunctionDatabase
      Initialize (a connection with) the database. If initialization is not successful, this routine will return false and an error description can be obtained using getLastError
      Specified by:
      initialize in interface FunctionDatabase
      Returns:
      true if the database ready for querying

    • storeSignatureRecord

      protected abstract long storeSignatureRecord(SignatureRecord sigrec) throws SQLException
      Make sure the vector corresponding to the SignatureRecord is inserted into the vectable
      Parameters:
      sigrec - is the SignatureRecord
      Returns:
      the computed id of the vector
      Throws:
      SQLException - if there is a problem creating or executing the query

    • retrieveFuncDescFromVectors

      protected int retrieveFuncDescFromVectors(VectorResult dresult, DescriptionManager res, int count, QueryNearest query, BSimSqlClause filter, SimilarityResult simres) throws SQLException, LSHException
      Generates FunctionDescription objects for each function with signature vector dresult. If a non-null filter is provided, FunctionDescriptions are only created for functions with pass the filter.
      Parameters:
      dresult - VectorResult representing a matching vector
      res - DescriptionManager container for the results
      count - number of function descriptions which have already been returned for this base vector
      query - contains various thresholds for the query
      filter - specifies additional conditions functions (and exes) must meet after meeting sim/signif threshold
      simres - receives the list of results and their similarity to the base vector
      Returns:
      number of
      Throws:
      SQLException - if there is an error issuing the query
      LSHException - if executable record creation fails

    • queryNearestVector

      protected abstract int queryNearestVector(List<VectorResult> resultset, LSHVector vec, double simthresh, double sigthresh, int max) throws SQLException
      Parameters:
      resultset - the list of result set objects to populate
      vec - the vector containing the saveSQL query statement
      simthresh - the similarity threshold
      sigthresh - the confidence threshold
      max - the max number of results to return
      Returns:
      the number of results returned
      Throws:
      SQLException - if there is a problem creating or executing the query

    • queryNearestVector

      protected abstract void queryNearestVector(QueryNearestVector query) throws SQLException
      Perform nearest vector query based upon specified query which contains response record which should be filled-in with search results.
      Parameters:
      query - the nearest vector query to perform
      Throws:
      SQLException - if there is an error issuing the query

    • doQuery

      protected QueryResponseRecord doQuery(BSimQuery<?> query, Connection c) throws LSHException, SQLException, FunctionDatabase.DatabaseNonFatalException
      Throws:
      LSHException
      SQLException
      FunctionDatabase.DatabaseNonFatalException

    • query

      public QueryResponseRecord query(BSimQuery<?> query)
      Description copied from interface: FunctionDatabase
      Send a query to the database. The response is returned as a QueryResponseRecord. If this is null, an error has occurred and an error message can be obtained from getLastError
      Specified by:
      query in interface FunctionDatabase
      Parameters:
      query - an object describing the query
      Returns:
      the response object or null if there is an error

    • queryFunctions

      protected int queryFunctions(QueryNearest query, BSimSqlClause filter, ResponseNearest response, DescriptionManager descMgr, Iterator<FunctionDescription> iter) throws SQLException, LSHException
      Parameters:
      query - the query to execute
      filter - the function filter
      response - the response object
      descMgr - the executable descriptor
      iter - the function iterator
      Returns:
      the number of unique results found
      Throws:
      SQLException - if there is an error issuing the query
      LSHException - if there is a problem creating executable records

    • getFunctionTags

      public List<String> getFunctionTags()
      Returns a list of all function tags in the database.
      Returns:
      list of function tags