嵌入式NoSQL数据库RaptorDB 3.1.0发布

jieforest

How It Works



The diagram above shows a high level view of how RaptorDB works, a document is first inserted into the storage file then immediately a primary map function is called to generate a primary view for the document at which point control returns to the calling method. After a timer has elapsed other map functions are called to generate other views for that document.

jieforest

There some term which you must be aware of when using RaptorDB document data store, these are the following:

1) Document : is an object/entity which gets serialized as JSON

2) DocID : is a Guid which uniquely identifies a document and must be given with the document

3) View : is a like a standard database table

4) Map Function : is a function that takes a document and emits values from that document into a view

5) Index : is used to retrieve information when querying views.

jieforest

What's a View?

A view is a list of values from a document which is bound to that document via a GUID property from that document. You can imagine a view as a 2 dimensional image of a multi dimensional document object usually created for a specific purpose.

For example if you have a invoice document one view would be a list of those invoices for the purpose of browsing them like:

1. { invoice GUID, date, invoice number, status, salesman name }

复制代码

Another view would be like below for the accounting department :

1. { invoice GUID, date, total sales amount, total sales discounts, salesman name, customer name }

复制代码

jieforest

What's a map function?

A map function is a piece of code you write to take a document object and "emit" a list of values usually from that document (your are free to emit anything, but most times it will be somehow related to that document) to a "view".

What's a "Primary View" for?

A primary view is a view which will get it's map function called immediately after a document save and there is no delay. This is so you can get a list of those documents immediately and don't have to wait. To show the importance of this take the following example :

You have a sales application and sales persons add their invoices, you would want to see these invoices after a save for the simple reason that you want to start the sales workflow process on that item, so you have to see it and you don't want to wait for a map function to fire "eventually".

Obviously the map function should be minimal and only show what is needed, you can go overboard and emit a lot of data for this list but you would loose performance.

jieforest

The Main Cast of Characters in RaptorDB

RaptorDB has the following parts:

1) RaptorDB main interface : what your code sees and the main loader of other parts.

2) Storage File : a fast and multi-threaded file storage mechanism used for both documents storage and view data.

3) View : see below

4) ViewManager : see below

5) ViewHandler : see below

jieforest

ViewHandler



The View is the responsible for

1) Storing view data

2) Indexing columns based on query use

jieforest

ViewManager

The View Manager is responsible for the following :

1）Loading and creating Views
2）Compiling view map functions.
3）Keeping track of mappings between the object types and views, or which view to send the object for inserting/updating.
4）Keeping track of last documents sent to views and the storage record numbers.
5）Periodic updating the views with new documents coming in

jieforest

Query Executor

The query executor will do the following :

1）Parse the LINQ expression.
2）Check the column names exist in the view schema, validation error will be caught here.
3）Check to see if indexes exist for the columns used in the filter
4）Extract the bitmap indexes needed.
5）Extract the Deleted records bitmap.
6）Execute the filter and fetch the view rows from the view storage.

RaptorDB Setup

For RaptorDB to function you must follow these steps:

Define a primary view for objects which will allow immediate access to inserted objects via a list, inherited objects are supported for simplicity so you can define one for the base class.

jieforest

The Save Process

The save process follows the following steps:

1）GetPrimaryListForType() :  will try to get the primary map function for the object type and recurs the hierarchy until found.
2）SaveData() : saves the data to storage file, creates log entry in the log file and in memory log, the background indexer will process the log.
3）SavePrimaryView() : will call the primary map function for this type.
4）SaveInOtherViews() : will save to any other view that is defined.

The process will fail to save if any one of the steps fail.

jieforest

Views and Map functions

A View is a list of data items from an object structure similar to a collection of rows in the table model. Map functions are responsible for taking an object and generating rows for views.

The premise behind map functions is that you precompute "queries" before-hand and just fetch the results when needed.

The map engine will do the following:

1）Find the last DocID processed and increment.

2）FindMapFunctionsForType() : a list of map functions to execute for this object

2.1）ExecuteMapFunction() : on the object and retrieve the new view rows with a reference to the DocID.
2.2）DeleteFromView(DocID) : flag delete the old values for DocID.
2.3）InsertView(newData) : add the new data from the map function to the view.

3）Find the next DocID and recurs the process.