Users input date as whatever they like, today they input 2023-4-28, tomorrow they forget what format they use before, they then input 4/29/2023.  Other users may input 30/4/2023,  or  04/30/2023, or 30-4-2023 or 04-30-2023 or just input month only 2023/4,  or just input year only 2023

 

https://transparentgov.net:3200/googlemaps141/default?layer_id=0&layer=NYPAD2_Easement&center_lat=41.18333803421051&center_long=-73.7791579159306&center_zoom=13&url=https%3A%2F%2Fservices6.arcgis.com%2FDZHaqZm9cxOD4CWM%2FArcGIS%2Frest%2Fservices%2FNYPAD2_Easement%2FFeatureServer&panto=0&overlayType=overlayType_none&select_field=14&select_fieldvalue=-1&distinctorgroupby=groupby

 

 

https://transparentgov.net:3200/googlemaps141/default?layer_id=0&layer=NYPAD_2_Fee&center_lat=43.07607440739406&center_long=-74.21026135147456&center_zoom=19&url=https%3A%2F%2Fservices6.arcgis.com%2FDZHaqZm9cxOD4CWM%2FArcGIS%2Frest%2Fservices%2FNYPAD_2_Fee%2FFeatureServer&panto=0&overlayType=overlayType_none&select_field=14&select_fieldvalue=-1&distinctorgroupby=groupby

 

This exactly happen in New York State protected land layers. 

Above example, source date column (Src_date) is text field, char(15). Not  esri 'date' type. This is very common in other states and county as well. They collect data from different source, date format is totally different is not unusual. 

 

 

I am trying to standardize these date format using Qgis 

 

Step 1) Download above web service layer into local hard drive. Search my other post on How To download arcgis web service layer use Qgis.

 

 

 

Step 2) add new field

 

 

 

 

 

 

 

 

 

 

 

Step 3) prepare python script

 ===================================================

 

 Qgis field calculator expression: 

 

           standardize_date( "Src_Date" )

 

 ------------------- python script for qgis only -------------------

 

from qgis.core import *

from qgis.gui import *

from datetime import datetime

import re

 

@qgsfunction(args='auto', group='Custom')

def standardize_date(text_field, feature, parent):

    

    #return re.sub('^2020/08/12$','999',text_field)  # test works

    #return re.sub('\d{4}/\d{2}/\d{2}','999',text_field)  # test works

    

    striped_raw_date_string = text_field.strip() 

 

# 1) find date format 2023-10-0

    result1 = re.findall('\d{4}-\d{2}-\d{2}',striped_raw_date_string) 

 

# 2) find date format 31/04/2023 or 31/4/2023  

    result2 = re.findall('(0?[1-9]|[12][0-9]|3[01])\/(0?[1-9]|1[012])\/\d{4}',striped_raw_date_string) 

 

    # 3) find date format 4/3/2020 or  04/03/2020   

    result3 = re.findall('^(0?[1-9]|1[012])\/(0?[1-9]|[12][0-9]|3[01])\/\d{4}',striped_raw_date_string) 

 

    # 4) find date format 2021/11/08

    result4 = re.findall('\d{4}\/(0?[1-9]|1[012])\/(0?[1-9]|[12][0-9]|3[01])',striped_raw_date_string) 

 

    # 5) find date format 2023 

    result5 = re.findall('^\d{4}$',striped_raw_date_string) 

 

    if (result1):

     date_time_obj = datetime.strptime(striped_raw_date_string, '%Y-%m-%d') 

     return date_time_obj.strftime("%Y/%m/%d")

    elif (result2):

     date_time_obj = datetime.strptime(striped_raw_date_string, '%d/%m/%Y')

     return date_time_obj.strftime("%Y/%m/%d")

    elif (result3):

     date_time_obj = datetime.strptime(striped_raw_date_string, '%m/%d/%Y')   

     return date_time_obj.strftime("%Y/%m/%d")

    elif (result4):

     date_time_obj = datetime.strptime(striped_raw_date_string, '%Y/%m/%d') 

     return date_time_obj.strftime("%Y/%m/%d")

    elif (result5):

     date_time_obj = datetime.strptime(striped_raw_date_string, '%Y')   

     return date_time_obj.strftime("%Y/%m/%d")

    else: 

    # empty or null set default valid date 

     date_time_obj = datetime.strptime('1900-01-01', '%Y-%m-%d')  

     return date_time_obj.strftime("%Y/%m/%d")

 

 

  

  ---------- end --------- qgis -------------------

 

  ================= test data ==================================

2020 

2019/10/16

2020/08/12 

31/01/2020 

2020-11-03 

05/01/2020 

04/17/2020 

2019/12/31 

2021-07-14

2019/12/04 

2010/12/16 

2021 

04/03/2020 

4/30/2020

2011/07/18 

2017/06/20 

2019 

2021-10-13 

2020

2021-07-14 

2021/11/08 

2020-11-03 

2021 

2019/12/31

2019-12-04 

null

2020/01/04

 

 

 

 

 

Step 4) open field calculator

edit python function expression

 

 

 

 

 

Step 5) implement python function

 

 

 

 

 

 

6) final result

 

 

 

reference

Working with Regular Expression in QGIS

https://umar-yusuf.blogspot.com/2020/08/regular-expression-in-qgis.html

 

 

Qgis document

https://docs.qgis.org/3.28/en/docs/user_manual/expressions/functions_list.html#expression-function-string-regexp-match

 

Test regex at here    https://regex101.com

by

Please log in or register