Package ghidra.features.bsim.query.elastic

Class ElasticDatabase

java.lang.Object

ghidra.features.bsim.query.elastic.ElasticDatabase

All Implemented Interfaces:

FunctionDatabase, AutoCloseable

public class ElasticDatabase
extends Object
implements FunctionDatabase

Implement the BSim database interface on top of an ElasticSearch back-end ElasticSearch holds records as JSON documents. Documents are stored in a specific "index". The primary BSim document index/types are: executable/exe is executable metadata corresponding to the ExecutableRecord object executable/function is function metadata corresponding to the FunctionDescription object vector/vector is the main feature vector corresponding to an LSHVector object meta/meta is a document containing the duplication count for a particular feature vector

Nested Class Summary

Nested classes/interfaces inherited from interface ghidra.features.bsim.query.FunctionDatabase

FunctionDatabase.BSimError, FunctionDatabase.ConnectionType, FunctionDatabase.DatabaseNonFatalException, FunctionDatabase.ErrorCategory, FunctionDatabase.Status

Field Summary

Fields

Modifier and Type	Field	Description
static final int	LAYOUT_VERSION
static final int	MAX_FUNCTION_BULK
static final int	MAX_FUNCTION_WINDOW
static final int	MAX_FUNCTIONUPDATE_WINDOW
static final int	MAX_VECTOR_BULK
static final int	MAX_VECTOR_OVERALL
static final int	MAX_VECTORCOUNT_WINDOW
static final int	MAX_VECTORDELETE_WINDOW

Constructor Summary

Constructors

Constructor	Description
ElasticDatabase(URL baseURL)	Construct the database connection given a URL.

Method Summary

Modifier and Type	Method	Description
void	close()	Close down (the connection with) the database
int	compareLayout()	Return -1 if info layout version is earlier than current client expectation Return 1 if info layout version is later than current client expectation Return 0 if info version and client version are the same
protected void	convertDescriptionRows(SimilarityResult similarityResult, com.google.gson.JsonArray descRows, VectorResult vectorResult, DescriptionManager manager, SignatureRecord sigRecord)	Convert function documents, presented as an array of JSON objects, that all share a single feature vector returned by a nearest neighbor query, into FunctionDescriptions and a full SimilarityResult.
protected int	countExecutables(String filter)	Place the same query for executables as queryExecutables(DescriptionManager, List, int, String, boolean, String).
static String	escape(String s)
FunctionDatabase.ConnectionType	getConnectionType()
DatabaseInformation	getInfo()
FunctionDatabase.BSimError	getLastError()	If the last query failed to produce a response, use this method to recover the error message
LSHVectorFactory	getLSHVectorFactory()
BSimServerInfo	getServerInfo()	Return the server info object for this database
FunctionDatabase.Status	getStatus()
String	getURLString()
String	getUserName()
boolean	initialize()	Initialize (a connection with) the database.
boolean	isInitialized()
QueryResponseRecord	query(BSimQuery<?> query)	Send a query to the database.
String	recoverExternalFunctionId(String exeName, String funcName, String arch)	Given the name of an executable library, its architecture, and a function name, return the id of the document describing this specific function.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface ghidra.features.bsim.query.FunctionDatabase

changePassword, getOverviewFunctionsPerStage, getQueriedFunctionsPerStage, isPasswordChangeAllowed

Field Details

LAYOUT_VERSION

public static final int LAYOUT_VERSION

See Also:

• Constant Field Values

MAX_VECTOR_OVERALL

public static final int MAX_VECTOR_OVERALL

See Also:

• Constant Field Values

MAX_FUNCTION_WINDOW

public static final int MAX_FUNCTION_WINDOW

See Also:

• Constant Field Values

MAX_FUNCTIONUPDATE_WINDOW

public static final int MAX_FUNCTIONUPDATE_WINDOW

See Also:

• Constant Field Values

MAX_VECTORCOUNT_WINDOW

public static final int MAX_VECTORCOUNT_WINDOW

See Also:

• Constant Field Values

MAX_VECTORDELETE_WINDOW

public static final int MAX_VECTORDELETE_WINDOW

See Also:

• Constant Field Values

MAX_FUNCTION_BULK

public static final int MAX_FUNCTION_BULK

See Also:

• Constant Field Values

MAX_VECTOR_BULK

public static final int MAX_VECTOR_BULK

See Also:

• Constant Field Values

Constructor Details

ElasticDatabase

public ElasticDatabase(URL baseURL)
                throws MalformedURLException

Construct the database connection given a URL. The URL protocol must be http, and the URL path must contain exactly one element naming the particular repository on the server.

Parameters:

baseURL - is the http URL

Throws:

MalformedURLException - if the URL is malformed

Method Details

escape

public static String escape(String s)

countExecutables

protected int countExecutables(String filter)
                        throws ElasticException

Place the same query for executables as queryExecutables(DescriptionManager, List, int, String, boolean, String). Except we only return the count of matching records.

Parameters:

filter - is the option filter options for the count

Returns:

the number of matching executables matching the filter

Throws:

ElasticException - is there is a server-side issue with the query

convertDescriptionRows

protected void convertDescriptionRows(SimilarityResult similarityResult,
 com.google.gson.JsonArray descRows,
 VectorResult vectorResult,
 DescriptionManager manager,
 SignatureRecord sigRecord)
                               throws ElasticException,
LSHException

Convert function documents, presented as an array of JSON objects, that all share a single feature vector returned by a nearest neighbor query, into FunctionDescriptions and a full SimilarityResult. Each function document will be parsed into a FunctionDescription, and a SimilarityNote will be created describing its similarity to the query vector based on the raw VectorResult data.

Parameters:

similarityResult - is the container for the SimilarityNotes

descRows - is the array of JSON function documents

vectorResult - is the raw vector query result

manager - is the container for new FunctionDescriptions

sigRecord - is the shared feature vector

Throws:

ElasticException - for communication problems with the server

LSHException - for problems adding new records to the container

isInitialized

public boolean isInitialized()

Returns:

true if a connection has been successfully initialized

getStatus

public FunctionDatabase.Status getStatus()

Specified by:

getStatus in interface FunctionDatabase

Returns:

the status of the current connection with this database

getConnectionType

public FunctionDatabase.ConnectionType getConnectionType()

Specified by:

getConnectionType in interface FunctionDatabase

Returns:

the type of connection

getUserName

public String getUserName()

Specified by:

getUserName in interface FunctionDatabase

Returns:

username (being used to establish connection)

getLSHVectorFactory

public LSHVectorFactory getLSHVectorFactory()

Specified by:

getLSHVectorFactory in interface FunctionDatabase

Returns:

factory the database is using to create LSHVector objects

getInfo

public DatabaseInformation getInfo()

Specified by:

getInfo in interface FunctionDatabase

Returns:

an information object giving general characteristics and descriptions of this database

compareLayout

public int compareLayout()

Description copied from interface: FunctionDatabase

Return -1 if info layout version is earlier than current client expectation Return 1 if info layout version is later than current client expectation Return 0 if info version and client version are the same

Specified by:

compareLayout in interface FunctionDatabase

Returns:

comparison of actual database layout with layout expected by client

getServerInfo

public BSimServerInfo getServerInfo()

Description copied from interface: FunctionDatabase

Return the server info object for this database

Specified by:

getServerInfo in interface FunctionDatabase

Returns:

the server info object

getURLString

public String getURLString()

Specified by:

getURLString in interface FunctionDatabase

initialize

public boolean initialize()

Description copied from interface: FunctionDatabase

Initialize (a connection with) the database. If initialization is not successful, this routine will return false and an error description can be obtained using getLastError

Specified by:

initialize in interface FunctionDatabase

Returns:

true if the database ready for querying

close

public void close()

Description copied from interface: FunctionDatabase

Close down (the connection with) the database

Specified by:

close in interface AutoCloseable

Specified by:

close in interface FunctionDatabase

getLastError

public FunctionDatabase.BSimError getLastError()

Description copied from interface: FunctionDatabase

If the last query failed to produce a response, use this method to recover the error message

Specified by:

getLastError in interface FunctionDatabase

Returns:

a String describing the error

query

public QueryResponseRecord query(BSimQuery<?> query)

Description copied from interface: FunctionDatabase

Send a query to the database. The response is returned as a QueryResponseRecord. If this is null, an error has occurred and an error message can be obtained from getLastError

Specified by:

query in interface FunctionDatabase

Parameters:

query - an object describing the query

Returns:

the response object or null if there is an error

recoverExternalFunctionId

public String recoverExternalFunctionId(String exeName,
 String funcName,
 String arch)
                                 throws ElasticException

Given the name of an executable library, its architecture, and a function name, return the id of the document describing this specific function. These 3 Strings are designed to uniquely identify a library function.

Parameters:

exeName - is the name of the executable

funcName - is the name of the function

arch - is the executable architecture

Returns:

the document id of the matching function

Throws:

ElasticException - if the function (the executable) doesn't exist