At the moment, I'm working on a ETL project using node.js based AWS Lambda.
To save some time, I'd appreciate it if someone could put together the T (Transform) portion for me while I'm busy writing the L (Load) portion.
I've defined the requirements below. Please
let me know how long it will take you to write the function, and how much (in BTC) you'd require as payment for the service.
tl;dr:Use the filter object to transform the data object. Find the appropriate translations and extractions in the filter object by matching with the key in the data object. Based on the filter extractions, delete columns from the "fields" AND "data" elements in the data object (preserving order of remaining values). Based on the filter translations, translate the "fields" values in the data object (preserving order).
I need you to write a javascript function
transform(filter,data,callback) { . . . }filter is an object defined as:
{
dataType1: {
translations: {
columnName1: "newColumName1",
columnName2: "newColumName2",
columnName3: "newColumnName3"
},
extractions: [ "columnName5" ]
},
dataType2: {
translations: {
columnName1: "newColumName1",
columnName2: "newColumName2",
columnName4: "newColumnName4"
},
extractions: [ "columnName3", "columnName5" ]
},
dataType3: {
translations: {
columnName2: "newColumName2",
columnName4: "newColumName4",
columnName5: "newColumnName5"
},
extractions: [ ]
},
dataType4: {
translations: {
},
extractions: [ "columnName1", "columnName3" ]
}
}
Where:
- datatype1, dataType2, dataType3 are arbitrary strings that could be any string.
- There will always be at least 1 data type key in the filter object, and never more than 500.
- translations and extractions are key words that will always exist in any data type object, but (as seen in the example) may have empty object or empty array as values.
- columnName1, columnName2, columnName3, columnName4, columnName5, columnName6, newColumnName1, newColumnName2, newColumnName3, newColumnName4, newColumnName5, newColumnName6 are arbitrary strings that could be any string.
data is an object defined as (for example):
{
"dataType2": {
"fields": [ "columnName1", "columnName2", "columnName3", "columnName4", "columnName5", "columnName6" ],
"data": [
[ 1, 32.4, "some data", null, "youGetTheIdea", "etc" ]
[ 5, 3.14, "more data", "even more", null, "etc"]
]
}
There will never be more than 1 dataType at a time in the data object when the transform function is called
It is
possible that data will not have any useful data in it. In that case it will be:
data = {
"dataType":{
"fields":null,
"data":[]
}
}
is a function:
function callback(err, data) { ... }
Parameters:
err (Error) - Set to null if the request is successful. Otherwise an error object indicating a problem.
data (Object) - the transformed data object. Set to null if an error occurs.
The transform() function needs to look up the data type key from the data object in the filter object to find the necessary translations and the extractions for that data type.
Any occurrence of a translations key in the fields array of the data object needs to be changed in the fields array to the corresponding translations value. The order of the elements in the array must be preserved.
Any occurrence of an element from the extractions array that occurs in the fields array needs to be removed from the fields array AND the values at the same offset in the data array elements must ALSO be removed. The order of the remaining elements must be preserved.
Given the above examples, the resulting data object after transformation would be:
{
"dataType2": {
"fields": [ "newColumName1", "newColumName2", "newColumnName4", "columnName6" ],
"data": [
[ 1, 32.4, null, "etc" ]
[ 5, 3.14, "even more", "etc"]
]
}
