Users input date as whatever they like, today they input 2023-4-28, tomorrow they forget what format they use before, they then input 4/29/2023.  Other users may input 30/4/2023,  or  04/30/2023, or 30-4-2023 or 04-30-2023 or just input month only 2023/4,  or just input year only 2023

 

https://transparentgov.net:3200/googlemaps141/default?layer_id=0&layer=NYPAD2_Easement&center_lat=41.18333803421051&center_long=-73.7791579159306&center_zoom=13&url=https%3A%2F%2Fservices6.arcgis.com%2FDZHaqZm9cxOD4CWM%2FArcGIS%2Frest%2Fservices%2FNYPAD2_Easement%2FFeatureServer&panto=0&overlayType=overlayType_none&select_field=14&select_fieldvalue=-1&distinctorgroupby=groupby

 

 

https://transparentgov.net:3200/googlemaps141/default?layer_id=0&layer=NYPAD_2_Fee&center_lat=43.07607440739406&center_long=-74.21026135147456&center_zoom=19&url=https%3A%2F%2Fservices6.arcgis.com%2FDZHaqZm9cxOD4CWM%2FArcGIS%2Frest%2Fservices%2FNYPAD_2_Fee%2FFeatureServer&panto=0&overlayType=overlayType_none&select_field=14&select_fieldvalue=-1&distinctorgroupby=groupby

 

This exactly happen in New York State protected land layers. 

Above example, source date column (Src_date) is text field, char(15). Not  esri 'date' type. This is very common in other states and county as well. They collect data from different source, date format is totally different is not unusual. 

 

 

I am trying to standardize these date format using Qgis 

 

Step 1) Download above web service layer into local hard drive. Search my other post on How To download arcgis web service layer 

 

Step 2) add new field

 

Step 3) prepare python script

ArcPro attribute table field calculator expression:

 

            new_field = standardize_date(!Src_Date!)

 

# ############# python script #############

 

from datetime import datetime

import re

def standardize_date(raw_date_string):

    #return re.sub('^2020/08/12$','999',raw_date_string)  # test works

    #return re.sub('\d{4}/\d{2}/\d{2}','999',raw_date_string)  # test works

 

striped_raw_date_string = raw_date_string.strip()

    

    # 1) find date format 2023-10-04

    result1 = re.findall('\d{4}-\d{2}-\d{2}', striped_raw_date_string)

 

# 2) find date format 31/04/2023 or 31/4/2023    

    result2 = re.findall('(0?[1-9]|[12][0-9]|3[01])\/(0?[1-9]|1[012])\/\d{4}', striped_raw_date_string)

    

    # 3) find date format 4/3/2020 or  04/03/2020   

    result3 = re.findall('^(0?[1-9]|1[012])\/(0?[1-9]|[12][0-9]|3[01])\/\d{4}', striped_raw_date_string)

    

    # 4) find date format 2021/11/08

    result4 = re.findall('\d{4}\/(0?[1-9]|1[012])\/(0?[1-9]|[12][0-9]|3[01])', striped_raw_date_string)

    

    # 5) find date format 2023    

    result5 = re.findall('^\d{4}$', striped_raw_date_string)

 

 

   

    if (result1):

     date_time_obj = datetime.strptime(striped_raw_date_string, '%Y-%m-%d')   

     return date_time_obj.strftime("%Y/%m/%d")

elif (result2):

     date_time_obj = datetime.strptime(striped_raw_date_string, '%d/%m/%Y')   

     return date_time_obj.strftime("%Y/%m/%d")

    elif (result3):

     date_time_obj = datetime.strptime(striped_raw_date_string, '%m/%d/%Y')   

     return date_time_obj.strftime("%Y/%m/%d")

    elif (result4):

     date_time_obj = datetime.strptime(striped_raw_date_string, '%Y/%m/%d')   

     return date_time_obj.strftime("%Y/%m/%d")

elif (result5):

     date_time_obj = datetime.strptime(striped_raw_date_string, '%Y')   

     return date_time_obj.strftime("%Y/%m/%d")

    else: 

# empty or null set default valid date 

     date_time_obj = datetime.strptime('1900-01-01', '%Y-%m-%d')   

     return date_time_obj.strftime("%Y/%m/%d")

 

# ##### end ######## python script #############

 

############## test data ##############

 

2020

2019/10/16

2020/08/12 

31/01/2020 

2020-11-03 

05/01/2020 

04/17/2020 

2019/12/31 

2021-07-14

2019/12/04 

2010/12/16 

2021

04/03/2020 

4/30/2020

2011/07/18 

2017/06/20 

2019

2021-10-13 

2020

2021-07-14 

2021/11/08 

2021/1/8 

2020-11-03 

2021

2019/12/31

2019-12-04 

null

2020/01/04

 

 

Step 4) python field calculator

 

5) final result

 

 

by

Please log in or register