Dataset management

Get a list of all datasets

GET /api/v2/organizations/:org_id/datasets
Authorization: Bearer ZW1haWxAZXhhbXBsZS5jb206cGFzc3dvcmQ=
Accept: application/json

The response should look like this:

{
    "datasets": [{
        "name": "Listing reviews 2012-2014",
        "description": "All listing reviews in English between 2012 and 2014",
        "numberOfDocuments": 18723,
        "createdAt": "2017-01-11T20:53:53.374Z",
        "fields": [{
            "name": "review_id",
            "isVisible": false,
            "isPrimaryKey": true,
            "analyze": false,
            "mappingName": "",
            "dataType": "Number"
        }, {
            "name": "review_date",
            "dateFormat": "MM-dd-yyyy",
            "isVisible": true,
            "isPrimaryKey": false,
            "analyze": false,
            "mappingName": "",
            "dataType": "Date"
        }, {
            "name": "review_content",
            "isVisible": true,
            "isPrimaryKey": false,
            "analyze": true,
            "mappingName": "124",
            "dataType": "String"
        }, {
            "name": "review_url",
            "isVisible": true,
            "isPrimaryKey": false,
            "analyze": false,
            "mappingName": "",
            "dataType": "URL"
        }],
        "id": "58769b6123e6480a9858ce8e"
    }]
}

Create a dataset

POST /api/v2/organizations/:org_id/datasets
Authorization: Bearer ZW1haWxAZXhhbXBsZS5jb206cGFzc3dvcmQ=
Content-Type: application/json
Accept: application/json

{
    "name": "Listing reviews 2012-2014",
    "description": "All listing reviews in English between 2012 and 2014",
    "fields": [{
        "dataType": "Number",
        "analyze": false,
        "name": "review_id",
        "isPrimaryKey": true
    }, {
        "dataType": "Date",
        "analyze": false,
        "name": "review_date",
        "dateFormat": "MM-DD-YYYY"
    }, {
        "dataType": "String",
        "analyze": true,
        "name": "review_content",
        "mappingName": "124",
    }, {
        "dataType": "URL",
        "analyze": false,
        "name": "review_url"
    }]
}
Property Type Required Description
name string yes A name displayed in the user interface.
description string no A description displayed under the dataset's name in the dataset grid.
fields array yes
Property Type Required Description
dataType string yes String, Number, Date, Boolean or URL.
analyze boolean no Set to true if the content has the type String and must be analyzed.
name string yes The name of the field.
mappingName string no An internal name for the field. The value of name is assigned by default.
dateFormat string yes* The date format. Required if dataType is Date and cannot be empty. See Supported date formats.
isPrimaryKey boolean no Set to true if the field value is a unique identifier. A dataset can only have one primary key.
isVisible boolean no Set to false to make the field invisible to users.

The response is a 200 OK status code:

200
{
    "id": "5703d9432327d444004e7c3d"
}
Property Description
id The ID associated to the new dataset.

Populate a dataset

POST /api/v2/datasets/:dataset_id/records
Authorization: Bearer ZW1haWxAZXhhbXBsZS5jb206cGFzc3dvcmQ=
Content-Type: application/json
Accept: application/json

{
    "records": [{
        "review_content": "Very nice and aesthetically pleasing accommodation.",
        "review_date": "10-06-2012",
        "review_url": {
            "target": "http://mysite.com/4055629"
            "label": "review #4055629"
        },
        "review_id": "4055629"
    }, {
        "review_content": "It was great for the price and the description accurate.",
        "review_date": "05-08-2012",
        "review_id": "41133812"
    }],
    "hasMore": false
}
Property Type Required Description
records array yes A list of records with their metadata. All fields must have been defined previously.
hasMore boolean yes Set to false if pushing your last records. Triggers post-processing.

The response should look like this:

{
    "failed": [
         "123456",
         "986464",
         "aslkdj"
    ],
    "enqueued": 97,
    "processed": 1000
}
Property Type Description
failed array A list of records that failed to parse, identified by their primary key.
enqueued integer The number of documents currently enqueued for processing.
processed integer The number of documents already processed.

Update a dataset

PUT /api/v2/datasets/:dataset_id/records
Authorization: Bearer ZW1haWxAZXhhbXBsZS5jb206cGFzc3dvcmQ=
Content-Type: application/json
Accept: application/json

{
    "records": [{
        "review_content": "It wasn't great for the price and the description was inaccurate.",
        "review_date": "05-08-2012",
        "review_id": "41133812"
    }],
    "hasMore": false
}

The response should look like this:

{
    "failed": [],
    "enqueued": 12,
    "processed": 1000
}
Property Type Description
failed array A list of records that failed to parse, identified by their primary key.
enqueued integer The number of documents currently enqueued for processing.
processed integer The number of documents already processed.

Delete a dataset

DELETE /api/v2/datasets/:dataset_id
Authorization: Bearer ZW1haWxAZXhhbXBsZS5jb206cGFzc3dvcmQ=

The response is a 204 No Content status code if the dataset was deleted successfully.

results matching ""

    No results matching ""