# PREP 501: Ingesting JSON Test Data into Adobe Experience Platform In this tutorial, we will learn how to ingest test data especially nested data into the Platform. You will need this in order to do your Data Distiller modules. ## Prerequisites You need to setup DBVisualizer: {% content-ref url="unit-1-getting-started/prep-400-dbvisualizer-sql-editor-setup-for-data-distiller" %} [prep-400-dbvisualizer-sql-editor-setup-for-data-distiller](https://data-distilller.gitbook.io/adobe-data-distiller-guide/unit-1-getting-started/prep-400-dbvisualizer-sql-editor-setup-for-data-distiller) {% endcontent-ref %} You will need to download this JSON file. Extract the zip and copy the JSON file over: {% file src="" %} ## **Scenario** We are going to ingest LUMA data into our test environment. This is a [fictitious online store ](https://luma.enablementadobe.com/content/luma/us/en.html)created by Adobe

The fastest way to understand what is happening on the website is to check the Products tab. There are 3 categories of products for different (and all) personas. You can browse them. You authenticate yourself and also can add items to a cart. The data that we are ingesting into the Platform is the test website traffic data that conforms to the Adobe Analytics schema. Unlike the Movie Genre Targeting example where we simply dropped a CSV file and the data popped out as a dataset, we cannot do the same with JSON files as we need to specify the nested schema for the system to understand the schema of the data. ## Setup Azure Storage Explorer 1. We will be using an interesting technique to ingest this data which will also form the basis of simulating batch ingestion. Download the Azure Storage Explorer from this[ link](https://azure.microsoft.com/en-us/blog/microsoft-azure-data-lake-storage-adls-in-storage-explorer-public-preview/). Make sure you download the right version based on your OS and install it. 2. We will be using Azure Storage Explorer as a local file browser to upload files into AEP's Landing Zone: Azure-based blob storage that stays outside AEP's governance boundary. The Landing Zone is a TTL for data for 7 days and serves as a mechanism for teams to push data asynchronously into this staging zone prior to ingestion. It also is a fantastic tool for testing the ingestion of test data into AEP. 3. In the Azure Storage Explorer, open up the Connect Dialog by clicking the plug icon and then click on the ADLSGen2 container or directory option:

5. Choose the connection type as **Shared SAS URL**. What this means is that if there are multiple users who have access to the Landing Zone URL, they could all write over each other. If you are seeking isolation, it is only available at the sandbox level. There is one Landing Zone per sandbox.

6. Name the container and then add the credentials by going into **Adobe Experience Platform->Sources->Data Landing Zone**.

7. Now go into **Adobe Experience Platform UI->Sources->Catalog->Cloud Storage->Data Landing Zone** and **View Credentials**:

8. If you click on the View Credentials, you should get this screen. Click to copy the ***SAS Uri***

9. Copy the **SAS URI** into the Storage Explorer Account setup:

10. Click next to complete the setup:

11. The screen will look like the following. Either **drag and drop** the JSON file or **Upload**:

12. Navigate to **Adobe Experience Platform UI->Sources->Catalog->Cloud Storage->Data Landing Zone**. You will either see **Add Data** or **Setup** button on the card itself. Click it to access the data landing Zone.

13. Voila! You should now see the JSON file you uploaded. You will also be able to preview the first 8 to 10 records (top of the file) as well. These records will be used for validating our pipeline for ingestion later.

## Create a XDM Schema 1. Create a XDM Schema by going to **Adobe Experience Platform UI->Schemas->Create XDM Experience Event**

2. On the Schema screen, click on the pane for **Field groups->Add**

3. Search for "Adobe Analytics" as a term for Field Groups:

4. Add **Adobe Analytics ExperienceEvent Template** field group. This is a comprehensive field group but we will be using a portion of all the fields.

5. Save the schema as **Luma Web Data.**

## Ingest Data from Data Landing Zone 1. Click on the XDM Compliant dropdown and change it to **Yes**:

2. Go to the next screen and fill out the details as exactly shown in the screen below. Name the dataset as **luma\_web\_data,** choose the **Luma Web Dataset** schema, **and** enable **Partial Ingestion.**

3. Configure the **Scheduling** to **Minute** and for every **15 minutes.**

4. Click **Next** and **Finish**. Your dataflow should execute and you should see the dataset **luma\_web\_data** in **Adobe Experience Platform UI->Datasets.** Click on the dataset **luma\_web\_data.** You should see about 733K records ingested.

{% hint style="info" %} **Note:** By marking the dataset as XDM compatible in the dataflow step, we avoided having to go through a mapping process. I was able to choose XDM compatible because the Adobe Analytics schema I chose was a superset of the Luma schema. There is no point in me doing a manual mapping. If you are bringing in Adobe Analytics data in practice, you may not be this lucky as you will need to account for eVars and will need to do the mapping. That is beyond the scope of this guide. {% endhint %} ## Query the Data 1. The first query that you can type is: ```sql select * from luma_web_data; ```

2. To get 50,000 results, you need to configure DBvIsualizer. {% content-ref url="unit-1-getting-started/prep-400-dbvisualizer-sql-editor-setup-for-data-distiller" %} [prep-400-dbvisualizer-sql-editor-setup-for-data-distiller](https://data-distilller.gitbook.io/adobe-data-distiller-guide/unit-1-getting-started/prep-400-dbvisualizer-sql-editor-setup-for-data-distiller) {% endcontent-ref %} 3. If you need to query the complex object, say, for the web object, use the `to_json` construct ```sql select to_json(web) from luma_web_data; ```