Automated Publishing
Data.gov.au has built-in capabilities to enable custodians processes in automating the publication of datasets through either:
- Harvesting of own government portal (see Harvested Portals)
- Utilising the API
Contact the support team at support@data.gov.au to discuss setting up a harvest schedule.
Documentation on the API can be found within the official CKAN API documentation as this is the Data Management System data.gov.au is built upon.
Before commencing your own development of an application to hook into our API, it is recommended to do the following:
- Create an API Token
- Navigate to Profile and select API Tokens
- Enter a name for the API Token
- Select Create API Token
- The system will generate you an API Token and display on the screen
Please note: this API Token will only appear once and cannot be viewed again.
- Manually publish the dataset once to retrieve the URL address
API Examples
cURL is a command line tool and library to assist in data transfer. If you need to download the tool please navigate to the Official curl - Download page. From there you can select the version best suited for your operating system.
Documentation
For help relating to cURL please use the curl - Documentation Overview
Example
From the command line enter the following
curl -H 'Authorization: APIKEY' 'https://data.gov.au/data/api/action/resource_patch' --form upload=@FILEPATH --form id=RESOURCE_ID --form url=URLWhere
- APIKEY is your account API key.
- FILEPATH is the path to the file you wish to upload.
- RESOURCE_ID is the id of the resource you want to patch on Data.gov.au.
Documentation
For help relating to Microsoft Powershell please use the PowerShell Documentation - PowerShell | Microsoft Learn
5.1+ Example
By default Powershell 5.1 comes with Windows installations. As this version does not support Multipart form data the script builds its own and submits it with the request.
<# Get and set default proxy incase behind one #>
[System.Net.WebRequest]::DefaultWebProxy = [System.Net.WebRequest]::GetSystemWebProxy()
[System.Net.WebRequest]::DefaultWebProxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
<# Prerequisite variables #>
$apiKey = "API-KEY"
$resourceId = "RESOURCE-ID"
$uploadFilePath = "PATH"
<# Build metadata #>
$metadata = @{
description = "This was patched using the API"
}
<# Set the API call variable #>
$uri = "https://data.gov.au/data/api/action/resource_patch"
<# Build multipart form data #>
<# Multipart variables #>
$boundary = [System.Guid]::NewGuid().ToString();
$LF = "`r`n";
<# Multipart data #>
$data = [System.Collections.ArrayList]::new()
$data.Add("--$boundary")
$data.Add("Content-Disposition: form-data; name=`"id`"")
$data.Add("")
$data.Add($resourceId)
foreach($key in $metadata.Keys)
{
$data.Add("--$boundary")
$data.Add("Content-Disposition: form-data; name=`"$key`"")
$data.Add("")
$data.Add($metadata.$key)
}
$data.Add("--$boundary")
$data.Add("Content-Disposition: form-data; name=`"upload`"; filename=`"$uploadFilePath`"")
$data.Add("Content-Type: application/octet-stream$LF")
$data.Add("")
$data.Add($(Get-Content $uploadFilePath -Raw))
$data.Add("--$boundary--$LF")
$multipart = $data -join $LF
<# Build request header #>
$headers = @{
Authorization = $apiKey
}
<# Result variables #>
<# success stays true and message stays empty unless an error is encountered #>
$success = $true
$message = ""
<# Submit request and capture error if there is one #>
try {
Invoke-WebRequest -Uri $uri -Method POST -ContentType "multipart/form-data; boundary=`"$boundary`"" -Headers $Headers -Body $multipart
}
catch {
$Failure = $_.ErrorDetails.Message | ConvertFrom-Json
$success = $Failure.success
$message = $Failure.error.message
}
<# Return variables incase this script is called by another and want to pass the result #>
return $success, $message6.0+ Example
From Powershell 6.0 onwards, WebRequest supports the submittion of Form data which helps in a much more simple development.
<# Get and set default proxy incase behind one #>
[System.Net.WebRequest]::DefaultWebProxy = [System.Net.WebRequest]::GetSystemWebProxy()
[System.Net.WebRequest]::DefaultWebProxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
<# Prerequisite variables #>
$apiKey = "API-KEY"
$resourceId = "RESOURCE-ID"
$uploadFilePath = "PATH"
<# Set the API call variable #>
$uri = "https://data.gov.au/data/api/action/resource_patch"
<# Build form data #>
$data = @{
id = $resourceId
upload = Get-Item $uploadFilePath
}
<# Build request header #>
$headers = @{
Authorization = $apiKey
}
<# Result variables #>
<# success stays true and message stays empty unless an error is encountered #>
$success = $true
$message = ""
<# Submit request and capture error if there is one #>
try {
Invoke-WebRequest -Uri $uri -Method POST -Headers $Headers -Form $data
}
catch {
$Failure = $_.ErrorDetails.Message | ConvertFrom-Json
$success = $Failure.success
$message = $Failure.error.message
}
<# Return variables incase this script is called by another and want to pass the result #>
return $success, $messageFME is a data integration platform that excels in spatial data manipulation. Workspaces allow the user to visually understand the workflow and customize with a drag and drop system. The examples provided are basic and only achieve connecting to the Data.gov.au API. However, they are a great base line that you can build upon to meet your requirements.
Documentation
For help relating to FME please use the FME Support Center
Example
#! <?xml version="1.0" encoding="UTF-8" ?>
#! <WORKSPACE
# Command-line to run this workspace:
# "C:\Program Files\FME\fme.exe" ""
# --Upload_File ""
# --Resource_ID ""
#
#! ARCGIS_COMPATIBILITY="ARCGIS_AUTO"
#! ATTR_TYPE_ENCODING="SDF"
#! BEGIN_PYTHON=""
#! BEGIN_TCL=""
#! CATEGORY=""
#! DESCRIPTION=""
#! DESTINATION="NONE"
#! DESTINATION_ROUTING_FILE=""
#! DOC_EXTENTS="3063.89 1129.51"
#! DOC_TOP_LEFT="-224.118 -1180.39"
#! END_PYTHON=""
#! END_TCL=""
#! EXPLICIT_BOOKMARK_ORDER="false"
#! FME_BUILD_NUM="21302"
#! FME_DOCUMENT_GUID="5a75d4b4-8b23-4f56-80e4-6d809163901a"
#! FME_DOCUMENT_PRIORGUID="7b3b9f01-4858-4676-89b1-59a3becd4958,53097d62-1803-43b1-8e62-273b806134c5"
#! FME_GEOMETRY_HANDLING="Enhanced"
#! FME_IMPLICIT_CSMAP_REPROJECTION_MODE="Auto"
#! FME_NAMES_ENCODING="windows-1252"
#! FME_REPROJECTION_ENGINE="FME"
#! FME_SERVER_SERVICES=""
#! FME_STROKE_MAX_DEVIATION="0"
#! HISTORY=""
#! IGNORE_READER_FAILURE="No"
#! LAST_SAVE_BUILD="FME(R) 2021.0.0.0 (20210305 - Build 21302 - WIN64)"
#! LAST_SAVE_DATE="2022-04-08T08:57:14"
#! LOG_FILE=""
#! LOG_MAX_RECORDED_FEATURES="200"
#! MARKDOWN_DESCRIPTION=""
#! MARKDOWN_USAGE=""
#! MAX_LOG_FEATURES="200"
#! MULTI_WRITER_DATASET_ORDER="BY_ID"
#! PASSWORD=""
#! PYTHON_COMPATIBILITY="38"
#! REDIRECT_TERMINATORS="NONE"
#! SAVE_ON_PROMPT_AND_RUN="Yes"
#! SHOW_ANNOTATIONS="true"
#! SHOW_INFO_NODES="true"
#! SOURCE="NONE"
#! SOURCE_ROUTING_FILE=""
#! TERMINATE_REJECTED="YES"
#! TITLE=""
#! USAGE=""
#! USE_MARKDOWN=""
#! VIEW_POSITION="-428.245 0"
#! WARN_INVALID_XFORM_PARAM="Yes"
#! WORKSPACE_VERSION="1"
#! ZOOM_SCALE="108"
#! >
#! <DATASETS>
#! </DATASETS>
#! <DATA_TYPES>
#! </DATA_TYPES>
#! <GEOM_TYPES>
#! </GEOM_TYPES>
#! <FEATURE_TYPES>
#! </FEATURE_TYPES>
#! <FMESERVER>
#! </FMESERVER>
#! <GLOBAL_PARAMETERS>
#! <GLOBAL_PARAMETER
#! GUI_LINE="GUI FILENAME_MUSTEXIST_OR_ATTR Upload_File "" Choose file to upload"
#! DEFAULT_VALUE=""
#! IS_STAND_ALONE="true"
#! />
#! <GLOBAL_PARAMETER
#! GUI_LINE="GUI TEXT_EDIT_OR_ATTR Resource_ID FME_SYNTAX%FME Enter the ID of the resource to patch"
#! DEFAULT_VALUE=""
#! IS_STAND_ALONE="true"
#! />
#! <GLOBAL_PARAMETER
#! GUI_LINE="GUI IGNORE OPTIONAL DISCLOSUREGROUP Private_Group Authorization Private"
#! DEFAULT_VALUE="<Unused>"
#! IS_STAND_ALONE="true"
#! />
#! <GLOBAL_PARAMETER
#! GUI_LINE="GUI IGNORE OPTIONAL TEXT_EDIT_OR_ATTR Authorization FME_SYNTAX%FME API Key"
#! DEFAULT_VALUE="API<space>KEY<space>HERE"
#! IS_STAND_ALONE="true"
#! />
#! </GLOBAL_PARAMETERS>
#! <USER_PARAMETERS
#! FORM="eyJwYXJhbWV0ZXJzIjpbeyJhY2Nlc3NNb2RlIjoicmVhZCIsImZpbHRlcnMiOltdLCJpdGVtc1RvU2VsZWN0IjoiZmlsZXMiLCJuYW1lIjoiVXBsb2FkX0ZpbGUiLCJwcm9tcHQiOiJDaG9vc2UgZmlsZSB0byB1cGxvYWQiLCJyZXF1aXJlZCI6dHJ1ZSwic2VsZWN0TXVsdGlwbGUiOmZhbHNlLCJzdXBwb3J0ZWRWYWx1ZVR5cGVzIjpbImV4cHJlc3Npb24iLCJnbG9iYWxQYXJhbWV0ZXIiXSwidHlwZSI6ImZpbGUiLCJ2YWxpZGF0ZUV4aXN0ZW5jZSI6dHJ1ZSwidmFsdWVUeXBlIjoic3RyaW5nIn0seyJlZGl0b3IiOiJwbGFpbnRleHQiLCJuYW1lIjoiUmVzb3VyY2VfSUQiLCJwcm9tcHQiOiJFbnRlciB0aGUgSUQgb2YgdGhlIHJlc291cmNlIHRvIHVwZGF0ZSIsInJlcXVpcmVkIjp0cnVlLCJzdXBwb3J0ZWRWYWx1ZVR5cGVzIjpbImV4cHJlc3Npb24iLCJnbG9iYWxQYXJhbWV0ZXIiXSwidHlwZSI6InRleHQiLCJ2YWx1ZVR5cGUiOiJzdHJpbmdFbmNvZGVkIn0seyJoaWRkZW4iOnRydWUsIm5hbWUiOiJQcml2YXRlX0dyb3VwIiwicGFyYW1ldGVycyI6W3siZGVmYXVsdFZhbHVlIjoiQVBJPHNwYWNlPktFWTxzcGFjZT5IRVJFIiwiZWRpdG9yIjoicGxhaW50ZXh0IiwiaGlkZGVuIjp0cnVlLCJuYW1lIjoiQXV0aG9yaXphdGlvbiIsInByb21wdCI6IkFQSSBLZXkiLCJyZXF1aXJlZCI6ZmFsc2UsInN1cHBvcnRlZFZhbHVlVHlwZXMiOlsiZXhwcmVzc2lvbiIsImdsb2JhbFBhcmFtZXRlciJdLCJ0eXBlIjoidGV4dCIsInZhbHVlVHlwZSI6InN0cmluZ0VuY29kZWQifV0sInByb21wdCI6IlByaXZhdGUiLCJyZXF1aXJlZCI6ZmFsc2UsInN0eWxlIjoiY29sbGFwc2libGUiLCJ0eXBlIjoiZ3JvdXAiLCJ2YWx1ZVR5cGUiOiJzdHJpbmcifV19"
#! >
#! <PARAMETER_INFO>
#! <INFO NAME="Authorization" IS_STAND_ALONE="true" />
#! <INFO NAME="Private_Group" IS_STAND_ALONE="true" />
#! <INFO NAME="Resource_ID" IS_STAND_ALONE="true" />
#! <INFO NAME="Upload_File" IS_STAND_ALONE="true" />
#! </PARAMETER_INFO>
#! </USER_PARAMETERS>
#! <COMMENTS>
#! <COMMENT
#! IDENTIFIER="7"
#! COMMENT_VALUE="<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-html40/strict.dtd"> <html><head><meta name="qrichtext" content="1" /><style type="text/css"> p, li { white-space: pre-wrap; } </style></head><body style=" font-family:'MS Shell Dlg 2'; font-size:9pt; font-weight:400; font-style:normal;"> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">Sends a POST request to the following API URL: <a href="https://data.gov.au/data/api/action/resource_patch"><span style=" text-decoration: underline;">https://data.gov.au/data/api/action/resource_patch</span></a> </p> <p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><br /></p> <p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><br /></p> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">In this request it has:</p> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">HEADER</p> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">	Authorization: API_KEY</p> <p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><br /></p> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">BODY</p> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">	Multipart/form-data</p> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">		upload: UPLOAD_FILE</p> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">		id: RESOURCE_ID</p> <p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><br /></p> <p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><br /></p> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">If you want to pass extra metadata values you can add additional attributes to the body of this HTTPCaller</p></body></html>"
#! POSITION="1373.6874312493119 -50.876343763437717"
#! TOP_LEFT="1373.6874312493119 -50.876343763437717"
#! BOTTOM_RIGHT="2378.1487814878142 -805.88359383593843"
#! BOUNDING_RECT="1373.6874312493119 -50.876343763437717 1004.4613502385023 755.00725007250071"
#! ORDER="500000000000008"
#! FOLLOW_ANCHOR="true"
#! INFO_NODE="false"
#! CUSTOM_USER_COLOR="false"
#! COLOUR="1,1,0.95686274509803926,0.78431372549019607"
#! SIZE_POLICY="10"
#! ANCHORED_NODE="3"
#! />
#! <COMMENT
#! IDENTIFIER="8"
#! COMMENT_VALUE="<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-html40/strict.dtd"> <html><head><meta name="qrichtext" content="1" /><style type="text/css"> p, li { white-space: pre-wrap; } </style></head><body style=" font-family:'MS Shell Dlg 2'; font-size:9pt; font-weight:400; font-style:normal;"> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><span style=" font-size:14pt;">Before running this workspace please enter your API key under the Private group within User Parameters.</span></p> <p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; font-size:14pt;"><br /></p> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><span style=" font-size:14pt;">File to upload and the resource id will be prompted upon workspace run.</span></p></body></html>"
#! POSITION="-224.11840280018959 -153.93924307929956"
#! TOP_LEFT="-224.11840280018959 -153.93924307929956"
#! BOTTOM_RIGHT="1003.1222008874131 -640.19380562492506"
#! BOUNDING_RECT="-224.11840280018959 -153.93924307929956 1227.2406036876027 486.25456254562545"
#! ORDER="500000000000011"
#! FOLLOW_ANCHOR="true"
#! INFO_NODE="false"
#! CUSTOM_USER_COLOR="false"
#! COLOUR="1,1,0.95686274509803926,0.78431372549019607"
#! SIZE_POLICY="10"
#! ANCHORED_NODE="9"
#! />
#! </COMMENTS>
#! <CONSTANTS>
#! </CONSTANTS>
#! <BOOKMARKS>
#! </BOOKMARKS>
#! <TRANSFORMERS>
#! <TRANSFORMER
#! IDENTIFIER="3"
#! TYPE="HTTPCaller"
#! VERSION="3"
#! POSITION="1678.1417814178139 -962.50962509625106"
#! BOUNDING_RECT="1678.1417814178139 -962.50962509625106 454 71"
#! ORDER="500000000000003"
#! PARMS_EDITED="true"
#! ENABLED="true"
#! LAST_PARM_EDIT="21302"
#! >
#! <OUTPUT_FEAT NAME="Output"/>
#! <FEAT_COLLAPSED COLLAPSED="0"/>
#! <XFORM_ATTR ATTR_NAME="_creation_instance" IS_USER_CREATED="false" FEAT_INDEX="0" />
#! <XFORM_ATTR ATTR_NAME="_response_body" IS_USER_CREATED="false" FEAT_INDEX="0" />
#! <XFORM_ATTR ATTR_NAME="_http_status_code" IS_USER_CREATED="false" FEAT_INDEX="0" />
#! <OUTPUT_FEAT NAME="<REJECTED>"/>
#! <FEAT_COLLAPSED COLLAPSED="1"/>
#! <XFORM_ATTR ATTR_NAME="_creation_instance" IS_USER_CREATED="false" FEAT_INDEX="1" />
#! <XFORM_ATTR ATTR_NAME="_error" IS_USER_CREATED="false" FEAT_INDEX="1" />
#! <XFORM_ATTR ATTR_NAME="fme_rejection_code" IS_USER_CREATED="false" FEAT_INDEX="1" />
#! <XFORM_ATTR ATTR_NAME="_response_body" IS_USER_CREATED="false" FEAT_INDEX="1" />
#! <XFORM_ATTR ATTR_NAME="_http_status_code" IS_USER_CREATED="false" FEAT_INDEX="1" />
#! <XFORM_PARM PARM_NAME="ADDITIONAL_PARAMETERS_GROUP" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="ADDITIONAL_URL_PARAMETERS" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="ADVANCED_REQUEST_PARAMETERS" PARM_VALUE="FME_DISCLOSURE_CLOSED"/>
#! <XFORM_PARM PARM_NAME="ADVANCED_RESPONSE_PARAMETERS" PARM_VALUE="NO"/>
#! <XFORM_PARM PARM_NAME="AUTH_METHOD" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="AUTH_NAMED_CONNECTION" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="AUTH_PASSWORD" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="AUTH_USERNAME" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="CONCURRENCY_GROUP" PARM_VALUE="NO"/>
#! <XFORM_PARM PARM_NAME="CONNECTION_TIMEOUT_LENGTH" PARM_VALUE="60"/>
#! <XFORM_PARM PARM_NAME="CUSTOM_HEADERS" PARM_VALUE="Authorization;$(Authorization)"/>
#! <XFORM_PARM PARM_NAME="CUSTOM_HEADERS_GROUP" PARM_VALUE="FME_DISCLOSURE_CLOSED"/>
#! <XFORM_PARM PARM_NAME="ERROR_ATTR" PARM_VALUE="_error"/>
#! <XFORM_PARM PARM_NAME="FILE_PATH_ATTR" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="FILE_TYPE" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="FOLLOW_REDIRECTS" PARM_VALUE="DO_NOT_FOLLOW"/>
#! <XFORM_PARM PARM_NAME="HTTPURL_GROUP" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="HTTP_AUTH_GROUP" PARM_VALUE="NO"/>
#! <XFORM_PARM PARM_NAME="HTTP_METHOD" PARM_VALUE="POST"/>
#! <XFORM_PARM PARM_NAME="HTTP_RESPONSE_GROUP" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="LOOP_MESSAGE" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="MAX_HTTP_VERSION" PARM_VALUE="HTTP_2"/>
#! <XFORM_PARM PARM_NAME="MAX_TRANSFERS_IN_PROGRESS" PARM_VALUE="25"/>
#! <XFORM_PARM PARM_NAME="MULTIPARTS" PARM_VALUE="id StringUpload SOURCE_EXPRESSION;$(Resource_ID);MIME_TYPE;text<solidus>plain;FME_CONTROLLER_QUERY_FILE;transformers<solidus>dependencies<solidus>http<solidus>StringUpload.fme;FME_CONTROLLER_CHOICE;StringUpload upload FileUpload SOURCE_FILE;$(Upload_File);MIME_TYPE;application<solidus>octet-stream;FME_CONTROLLER_QUERY_FILE;transformers<solidus>dependencies<solidus>http<solidus>FileUpload.fme;FME_CONTROLLER_CHOICE;FileUpload"/>
#! <XFORM_PARM PARM_NAME="MULTIPART_GROUP" PARM_VALUE="FME_DISCLOSURE_OPEN"/>
#! <XFORM_PARM PARM_NAME="OUTPUT_DIRNAME" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="OUTPUT_FILENAME" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="REDIRECT_AUTH" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="RESPONSE_FILE_GROUP" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="RESPONSE_HEADER_LIST_ATTR" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="SAVE_FILE" PARM_VALUE="Attribute"/>
#! <XFORM_PARM PARM_NAME="SAVE_MULTI" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="STATUS_CODE_ATTR" PARM_VALUE="_http_status_code"/>
#! <XFORM_PARM PARM_NAME="TARGET_ATTR" PARM_VALUE="_response_body"/>
#! <XFORM_PARM PARM_NAME="TARGET_ATTR_ENCODING" PARM_VALUE="auto-detect"/>
#! <XFORM_PARM PARM_NAME="TARGET_URL" PARM_VALUE="https:<solidus><solidus>data.gov.au<solidus>data<solidus>api<solidus>action<solidus>resource_patch"/>
#! <XFORM_PARM PARM_NAME="TRANSFER_TIMEOUT_LENGTH" PARM_VALUE="90"/>
#! <XFORM_PARM PARM_NAME="TRANSFORMER_GROUP" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="UPLOAD_BODY" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="UPLOAD_CONTENT_TYPE" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="UPLOAD_FILE" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="UPLOAD_GROUP" PARM_VALUE="FME_DISCLOSURE_OPEN"/>
#! <XFORM_PARM PARM_NAME="UPLOAD_TYPE" PARM_VALUE="Multipart / Form Data"/>
#! <XFORM_PARM PARM_NAME="USE_COOKIES" PARM_VALUE="No"/>
#! <XFORM_PARM PARM_NAME="VERIFY_SSL_CERTIFICATES" PARM_VALUE="Yes"/>
#! <XFORM_PARM PARM_NAME="XFORMER_NAME" PARM_VALUE="HTTPCaller"/>
#! </TRANSFORMER>
#! <TRANSFORMER
#! IDENTIFIER="4"
#! TYPE="Logger"
#! VERSION="4"
#! POSITION="2385.7679376793762 -1109.3860938609387"
#! BOUNDING_RECT="2385.7679376793762 -1109.3860938609387 454 71"
#! ORDER="500000000000004"
#! PARMS_EDITED="true"
#! ENABLED="true"
#! LAST_PARM_EDIT="21302"
#! >
#! <OUTPUT_FEAT NAME="LOGGED"/>
#! <FEAT_COLLAPSED COLLAPSED="0"/>
#! <XFORM_ATTR ATTR_NAME="_creation_instance" IS_USER_CREATED="false" FEAT_INDEX="0" />
#! <XFORM_ATTR ATTR_NAME="_error" IS_USER_CREATED="false" FEAT_INDEX="0" />
#! <XFORM_ATTR ATTR_NAME="fme_rejection_code" IS_USER_CREATED="false" FEAT_INDEX="0" />
#! <XFORM_ATTR ATTR_NAME="_response_body" IS_USER_CREATED="false" FEAT_INDEX="0" />
#! <XFORM_ATTR ATTR_NAME="_http_status_code" IS_USER_CREATED="false" FEAT_INDEX="0" />
#! <XFORM_PARM PARM_NAME="FEATURE_TYPE" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="LOG_FEATURE" PARM_VALUE="LOG_RECORD"/>
#! <XFORM_PARM PARM_NAME="MAXCOORDS" PARM_VALUE="20"/>
#! <XFORM_PARM PARM_NAME="MAXFEATS" PARM_VALUE="20"/>
#! <XFORM_PARM PARM_NAME="MESSAGE" PARM_VALUE="Feature<space>is:"/>
#! <XFORM_PARM PARM_NAME="MSG_GRP" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="PARAMETERS_GROUP" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="SEVERITY" PARM_VALUE="INFO"/>
#! <XFORM_PARM PARM_NAME="TRANSFORMER_GROUP" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="XFORMER_NAME" PARM_VALUE="Logger"/>
#! </TRANSFORMER>
#! <TRANSFORMER
#! IDENTIFIER="9"
#! TYPE="Creator"
#! VERSION="5"
#! POSITION="855.43784730776611 -798.61909730208424"
#! BOUNDING_RECT="855.43784730776611 -798.61909730208424 430 71"
#! ORDER="500000000000010"
#! PARMS_EDITED="true"
#! ENABLED="true"
#! LAST_PARM_EDIT="21302"
#! >
#! <OUTPUT_FEAT NAME="CREATED"/>
#! <FEAT_COLLAPSED COLLAPSED="0"/>
#! <XFORM_ATTR ATTR_NAME="_creation_instance" IS_USER_CREATED="false" FEAT_INDEX="0" />
#! <XFORM_PARM PARM_NAME="ATEND" PARM_VALUE="no"/>
#! <XFORM_PARM PARM_NAME="COORDS" PARM_VALUE="<Unused>"/>
#! <XFORM_PARM PARM_NAME="COORDSYS" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="CRE_ATTR" PARM_VALUE="_creation_instance"/>
#! <XFORM_PARM PARM_NAME="GEOM" PARM_VALUE="<lt>?xml<space>version=<quote>1.0<quote><space>encoding=<quote>US_ASCII<quote><space>standalone=<quote>no<quote><space>?<gt><lt>geometry<space>dimension=<quote>2<quote><gt><lt>null<solidus><gt><lt><solidus>geometry<gt>"/>
#! <XFORM_PARM PARM_NAME="GEOMTYPE" PARM_VALUE="Geometry Object"/>
#! <XFORM_PARM PARM_NAME="NUM" PARM_VALUE="1"/>
#! <XFORM_PARM PARM_NAME="PARAMETERS_GROUP" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="TRANSFORMER_GROUP" PARM_VALUE=""/>
#! <XFORM_PARM PARM_NAME="XFORMER_NAME" PARM_VALUE="Creator"/>
#! </TRANSFORMER>
#! </TRANSFORMERS>
#! <FEAT_LINKS>
#! <FEAT_LINK
#! IDENTIFIER="10"
#! SOURCE_NODE="9"
#! TARGET_NODE="3"
#! SOURCE_PORT_DESC="fo 0 CREATED"
#! TARGET_PORT_DESC="fi 0 "
#! ENABLED="true"
#! EXECUTION_IDX="0"
#! HIDDEN="false"
#! COLOUR="0,0,0,1"
#! PEN_WIDTH="4"
#! PEN_STYLE="1"
#! LINK_STYLE_NAME="<Default>"
#! EXTRA_POINTS=""
#! />
#! <FEAT_LINK
#! IDENTIFIER="5"
#! SOURCE_NODE="3"
#! TARGET_NODE="4"
#! SOURCE_PORT_DESC="fo 0 Output"
#! TARGET_PORT_DESC="fi 0 "
#! ENABLED="true"
#! EXECUTION_IDX="0"
#! HIDDEN="false"
#! COLOUR="0,0,0,1"
#! PEN_WIDTH="4"
#! PEN_STYLE="1"
#! LINK_STYLE_NAME="<Default>"
#! EXTRA_POINTS=""
#! />
#! <FEAT_LINK
#! IDENTIFIER="6"
#! SOURCE_NODE="3"
#! TARGET_NODE="4"
#! SOURCE_PORT_DESC="fo 1 <lt>REJECTED<gt>"
#! TARGET_PORT_DESC="fi 0 "
#! ENABLED="true"
#! EXECUTION_IDX="0"
#! HIDDEN="false"
#! COLOUR="0,0,0,1"
#! PEN_WIDTH="4"
#! PEN_STYLE="1"
#! LINK_STYLE_NAME="<Default>"
#! EXTRA_POINTS=""
#! />
#! </FEAT_LINKS>
#! <BREAKPOINTS>
#! </BREAKPOINTS>
#! <ATTR_LINKS>
#! </ATTR_LINKS>
#! <SUBDOCUMENTS>
#! </SUBDOCUMENTS>
#! <LOOKUP_TABLES>
#! </LOOKUP_TABLES>
#! </WORKSPACE>
FME_PYTHON_VERSION 38
ARCGIS_COMPATIBILITY ARCGIS_AUTO
# ============================================================================
DEFAULT_MACRO Upload_File
DEFAULT_MACRO Resource_ID
DEFAULT_MACRO Private_Group <Unused>
DEFAULT_MACRO Authorization API<space>KEY<space>HERE
# ============================================================================
GUI FILENAME_EXISTING Upload_File "" Choose file to upload
GUI TEXT_EDIT Resource_ID FME_SYNTAX%FME Enter the ID of the resource to patch
# ============================================================================
USER_PARAMETERS eyJwYXJhbWV0ZXJzIjpbeyJhY2Nlc3NNb2RlIjoicmVhZCIsImZpbHRlcnMiOltdLCJpdGVtc1RvU2VsZWN0IjoiZmlsZXMiLCJuYW1lIjoiVXBsb2FkX0ZpbGUiLCJwcm9tcHQiOiJDaG9vc2UgZmlsZSB0byB1cGxvYWQiLCJyZXF1aXJlZCI6dHJ1ZSwic2VsZWN0TXVsdGlwbGUiOmZhbHNlLCJzdXBwb3J0ZWRWYWx1ZVR5cGVzIjpbImV4cHJlc3Npb24iLCJnbG9iYWxQYXJhbWV0ZXIiXSwidHlwZSI6ImZpbGUiLCJ2YWxpZGF0ZUV4aXN0ZW5jZSI6dHJ1ZSwidmFsdWVUeXBlIjoic3RyaW5nIn0seyJlZGl0b3IiOiJwbGFpbnRleHQiLCJuYW1lIjoiUmVzb3VyY2VfSUQiLCJwcm9tcHQiOiJFbnRlciB0aGUgSUQgb2YgdGhlIHJlc291cmNlIHRvIHVwZGF0ZSIsInJlcXVpcmVkIjp0cnVlLCJzdXBwb3J0ZWRWYWx1ZVR5cGVzIjpbImV4cHJlc3Npb24iLCJnbG9iYWxQYXJhbWV0ZXIiXSwidHlwZSI6InRleHQiLCJ2YWx1ZVR5cGUiOiJzdHJpbmdFbmNvZGVkIn0seyJoaWRkZW4iOnRydWUsIm5hbWUiOiJQcml2YXRlX0dyb3VwIiwicGFyYW1ldGVycyI6W3siZGVmYXVsdFZhbHVlIjoiQVBJPHNwYWNlPktFWTxzcGFjZT5IRVJFIiwiZWRpdG9yIjoicGxhaW50ZXh0IiwiaGlkZGVuIjp0cnVlLCJuYW1lIjoiQXV0aG9yaXphdGlvbiIsInByb21wdCI6IkFQSSBLZXkiLCJyZXF1aXJlZCI6ZmFsc2UsInN1cHBvcnRlZFZhbHVlVHlwZXMiOlsiZXhwcmVzc2lvbiIsImdsb2JhbFBhcmFtZXRlciJdLCJ0eXBlIjoidGV4dCIsInZhbHVlVHlwZSI6InN0cmluZ0VuY29kZWQifV0sInByb21wdCI6IlByaXZhdGUiLCJyZXF1aXJlZCI6ZmFsc2UsInN0eWxlIjoiY29sbGFwc2libGUiLCJ0eXBlIjoiZ3JvdXAiLCJ2YWx1ZVR5cGUiOiJzdHJpbmcifV19
# ============================================================================
INCLUDE [ if {{$(Upload_File)} == {}} { puts_real {Parameter 'Upload_File' must be given a value.}; exit 1; }; ]
INCLUDE [ if {{$(Resource_ID)} == {}} { puts_real {Parameter 'Resource_ID' must be given a value.}; exit 1; }; ]
#! START_HEADER
#! START_WB_HEADER
READER_TYPE MULTI_READER
WRITER_TYPE NULL
WRITER_KEYWORD NULL_DEST
NULL_DEST_DATASET null
#! END_WB_HEADER
#! START_WB_HEADER
#! END_WB_HEADER
#! END_HEADER
LOG_FILENAME "$(FME_MF_DIR)FME Upload Example.log"
LOG_APPEND NO
LOG_FILTER_MASK -1
LOG_MAX_FEATURES 200
LOG_MAX_RECORDED_FEATURES 200
FME_REPROJECTION_ENGINE FME
FME_IMPLICIT_CSMAP_REPROJECTION_MODE Auto
FME_GEOMETRY_HANDLING Enhanced
FME_STROKE_MAX_DEVIATION 0
FME_NAMES_ENCODING windows-1252
# -------------------------------------------------------------------------
MULTI_READER_CONTINUE_ON_READER_FAILURE No
# -------------------------------------------------------------------------
MACRO WORKSPACE_NAME FME Upload Example
MACRO FME_VIEWER_APP fmedatainspector
DEFAULT_MACRO WB_CURRENT_CONTEXT
# -------------------------------------------------------------------------
Tcl2 proc Creator_CoordSysRemover {} { global FME_CoordSys; set FME_CoordSys {}; }
MACRO Creator_XML NOT_ACTIVATED
MACRO Creator_CLASSIC NOT_ACTIVATED
MACRO Creator_2D3D 2D_GEOMETRY
MACRO Creator_COORDS <Unused>
INCLUDE [ if { {Geometry Object} == {Geometry Object} } { puts {MACRO Creator_XML *} } ]
INCLUDE [ if { {Geometry Object} == {2D Coordinate List} } { puts {MACRO Creator_2D3D 2D_GEOMETRY}; puts {MACRO Creator_CLASSIC *} } ]
INCLUDE [ if { {Geometry Object} == {3D Coordinate List} } { puts {MACRO Creator_2D3D 3D_GEOMETRY}; puts {MACRO Creator_CLASSIC *} } ]
INCLUDE [ if { {Geometry Object} == {2D Min/Max Box} } { set comment { We need to turn the COORDS which are minX minY maxX maxY into a full polygon list of coordinates }; set splitCoords [split [string trim {<Unused>}]]; if { [llength $splitCoords] > 4} { set trimmedCoords {}; foreach item $splitCoords { if { $item != {} } {lappend trimmedCoords $item} }; set splitCoords $trimmedCoords; }; if { [llength $splitCoords] != 4 } { error {Creator: Coordinate list is expected to be a space delimited list of four numbers as 'minx miny maxx maxy' - `<Unused>' is invalid}; }; set minX [lindex $splitCoords 0]; set minY [lindex $splitCoords 1]; set maxX [lindex $splitCoords 2]; set maxY [lindex $splitCoords 3]; puts "MACRO Creator_COORDS $minX $minY $minX $maxY $maxX $maxY $maxX $minY $minX $minY"; puts {MACRO Creator_2D3D 2D_GEOMETRY}; puts {MACRO Creator_CLASSIC *} } ]
FACTORY_DEF {$(Creator_XML)} CreationFactory FACTORY_NAME { Creator_XML_Creator } CREATE_AT_END { no } OUTPUT { FEATURE_TYPE _____CREATED______ @Geometry(FROM_ENCODED_STRING,<lt>?xml<space>version=<quote>1.0<quote><space>encoding=<quote>US_ASCII<quote><space>standalone=<quote>no<quote><space>?<gt><lt>geometry<space>dimension=<quote>2<quote><gt><lt>null<solidus><gt><lt><solidus>geometry<gt>) }
FACTORY_DEF {$(Creator_CLASSIC)} CreationFactory FACTORY_NAME { Creator_CLASSIC_Creator } $(Creator_2D3D) { $(Creator_COORDS) } CREATE_AT_END { no } OUTPUT { FEATURE_TYPE _____CREATED______ }
FACTORY_DEF {*} TeeFactory FACTORY_NAME { Creator_Cloner } INPUT FEATURE_TYPE _____CREATED______ @Tcl2(Creator_CoordSysRemover) @CoordSys() NUMBER_OF_COPIES { 1 } COPY_NUMBER_ATTRIBUTE { "_creation_instance" } OUTPUT { FEATURE_TYPE Creator_CREATED fme_feature_type Creator }
FACTORY_DEF * BranchingFactory FACTORY_NAME "Creator_CREATED Brancher -1 10" INPUT FEATURE_TYPE Creator_CREATED TARGET_FACTORY "$(WB_CURRENT_CONTEXT)_CREATOR_BRANCH_TARGET" MAXIMUM_COUNT None OUTPUT PASSED FEATURE_TYPE *
# -------------------------------------------------------------------------
FACTORY_DEF * TeeFactory FACTORY_NAME "$(WB_CURRENT_CONTEXT)_CREATOR_BRANCH_TARGET" INPUT FEATURE_TYPE * OUTPUT FEATURE_TYPE *
# -------------------------------------------------------------------------
FACTORY_DEF {*} HTTPFactory FACTORY_NAME { HTTPCaller } INPUT FEATURE_TYPE Creator_CREATED TARGET_URL { "https:<solidus><solidus>data.gov.au<solidus>data<solidus>api<solidus>action<solidus>resource_patch" } HTTP_METHOD { POST } SAVE_FILE { NO } OUTPUT_FILENAME { "<Unused>" } OUTPUT_DIRECTORY { "<Unused>" } FILE_EXTENSION { "<Unused>" } TARGET_ATTR { "_response_body" } FILE_PATH_ATTR { "<Unused>" } UPLOAD_FILE { "<Unused>" } UPLOAD_BODY { "<Unused>" } MULTIPART_UPLOAD { YES } UPLOAD_CONTENT_TYPE { "<Unused>" } COMBINED_MULTIPART_TABLE { id StringUpload SOURCE_EXPRESSION;@EvaluateExpression(FDIV,STRING_ENCODED,$(Resource_ID),HTTPCaller);MIME_TYPE;text<solidus>plain upload FileUpload SOURCE_FILE;@EvaluateExpression(FDIV,STRING_ENCODED,$(Upload_File$encode),HTTPCaller);MIME_TYPE;application<solidus>octet-stream } REQUEST_HEADER_TABLE { Authorization;@EvaluateExpression(FDIV,STRING_ENCODED,$(Authorization),HTTPCaller) } ADDITIONAL_URL_PARAMETERS { } AUTH_USERNAME { "<Unused>" } AUTH_PASSWORD { "<Unused>" } AUTH_METHOD { "<Unused>" } TARGET_ATTRIBUTE_ENCODING { auto-detect } RESPONSE_HEADER_LIST_ATTR { "" } STATUS_CODE_ATTR { "_http_status_code" } ERROR_ATTR { "_error" } CONNECTION_TIMEOUT_LENGTH { 60 } TRANSFER_TIMEOUT_LENGTH { 90 } FOLLOW_REDIRECTS { DO_NOT_FOLLOW } REDIRECT_AUTH { <Unused> } VERIFY_SSL_CERTIFICATES { Yes } NAMED_CONNECTION { "<Unused>" } USE_COOKIES { No } MAX_TRANSFERS_IN_PROGRESS { 25 } MAX_HTTP_VERSION { HTTP_2 } OUTPUT { OUTPUT FEATURE_TYPE HTTPCaller_Output } OUTPUT { REJECTED FEATURE_TYPE HTTPCaller_<REJECTED> }
# -------------------------------------------------------------------------
FACTORY_DEF {*} TeeFactory FACTORY_NAME { Logger_Logger } INPUT FEATURE_TYPE HTTPCaller_Output INPUT FEATURE_TYPE HTTPCaller_<REJECTED> OUTPUT { FEATURE_TYPE Logger_LOGGED @Log(ENCODED, "Logger:<space>Feature<space>is:","20","20","",INFO,LOG_RECORD,Logger_2e587d81_001a_475c_bee4_bc874503ce060) }
FACTORY_DEF * TeeFactory FACTORY_NAME "Logger LOGGED Transformer Output Nuker" INPUT FEATURE_TYPE Logger_LOGGED
# -------------------------------------------------------------------------
FACTORY_DEF * RoutingFactory FACTORY_NAME "Destination Feature Type Routing Correlator" COMMAND_PARM_EVALUATION SINGLE_PASS INPUT FEATURE_TYPE * FEATURE_TYPE_ATTRIBUTE __wb_out_feat_type__ OUTPUT ROUTED FEATURE_TYPE * OUTPUT NOT_ROUTED FEATURE_TYPE __nuke_me__ @Tcl2("FME_StatMessage 818059 [FME_GetAttribute fme_template_feature_type] 818060 818061 fme_warn")
# -------------------------------------------------------------------------
FACTORY_DEF * TeeFactory FACTORY_NAME "Final Output Nuker" INPUT FEATURE_TYPE __nuke_me__