| Dependency | Reason |
|---|---|
| Task Instance State | Task is in the 'success' state which is not a valid state for execution. The task must be cleared in order to be run. |
| Dagrun Running | Task instance's dagrun was not in the 'running' state but in the state 'success'. |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | def getCNCEData(**context):
ti = context["ti"]
token = ti.xcom_pull(key="token")
response = requests.get(
url=f"{dRoW_api_end_url}/api/module/document-export/airflow/workflow/637c7d22b38f8ca02f5c49ab?export_type=0",
headers={
"x-access-token": f"Bearer {token}",
"ICWPxAccessKey": "5WSD21ICWP_[1AG:4UdI){n=b~"
}
)
cnceData = response.json()
print(f"check cnce data: {cnceData}")
today = datetime.now(timezone.utc)
rows = []
for item in cnceData:
d = (item.get("data") or {})
event_no = (d.get("NEC Event No.") or "").strip()
doc_type = (d.get("NEC Doc Type") or "").strip()
doc_date_raw = d.get("Doc Date") # boss wants this
receive_date_raw = d.get("Receive Date")
# pick a best-effort "doc_datetime"
doc_dt = parse_any_dt(doc_date_raw) or parse_any_dt(receive_date_raw)
if not event_no or not doc_type or not doc_dt:
continue
rows.append({
"event_no": event_no,
"doc_type": doc_type.strip().upper(),
"doc_dt": doc_dt,
})
# 2) group by event_no and find latest doc
latest_by_event = {}
for r in rows:
key = r["event_no"]
cur = latest_by_event.get(key)
if (cur is None) or (r["doc_dt"] > cur["doc_dt"]):
latest_by_event[key] = r
# 3) unreplied NCE = latest doc is NCE
unreplied_nce = []
for event_no, latest in latest_by_event.items():
if latest["doc_type"].startswith("NCE"):
unreplied_nce.append(latest) # keep latest doc row (its doc_dt is the NCE date)
# 4) KPI rules (calendar months)
kpi5_cutoff = today - relativedelta(months=3)
kpi6_cutoff = today - relativedelta(months=6)
kpi5 = sum(1 for r in unreplied_nce if r["doc_dt"] < kpi5_cutoff)
kpi6 = sum(1 for r in unreplied_nce if r["doc_dt"] < kpi6_cutoff)
print("unreplied NCE total:", len(unreplied_nce))
print("KPI 5:", kpi5)
print("KPI 6:", kpi6)
# push into XCom so writeKpiToPostgres can store
ti.xcom_push(key="kpi5", value=int(kpi5))
ti.xcom_push(key="kpi6", value=int(kpi6))
return {"kpi5": int(kpi5), "kpi6": int(kpi6)}
|
| Attribute | Value |
|---|---|
| dag_id | nd201905_kpi |
| duration | 155.091491 |
| end_date | 2025-12-16 15:10:44.635330+00:00 |
| execution_date | 2025-12-16T07:00:00+00:00 |
| executor_config | {} |
| generate_command | <function TaskInstance.generate_command at 0x7fb2f39d0320> |
| hostname | a7c46ba165e9 |
| is_premature | False |
| job_id | 4390 |
| key | ('nd201905_kpi', 'getCNCEData', <Pendulum [2025-12-16T07:00:00+00:00]>, 2) |
| log | <Logger airflow.task (INFO)> |
| log_filepath | /usr/local/airflow/logs/nd201905_kpi/getCNCEData/2025-12-16T07:00:00+00:00.log |
| log_url | http://localhost:8080/admin/airflow/log?execution_date=2025-12-16T07%3A00%3A00%2B00%3A00&task_id=getCNCEData&dag_id=nd201905_kpi |
| logger | <Logger airflow.task (INFO)> |
| mark_success_url | http://localhost:8080/success?task_id=getCNCEData&dag_id=nd201905_kpi&execution_date=2025-12-16T07%3A00%3A00%2B00%3A00&upstream=false&downstream=false |
| max_tries | 1 |
| metadata | MetaData(bind=None) |
| next_try_number | 2 |
| operator | PythonOperator |
| pid | 963352 |
| pool | default_pool |
| prev_attempted_tries | 1 |
| previous_execution_date_success | 2025-12-16 04:52:25.593744+00:00 |
| previous_start_date_success | 2025-12-16 04:55:25.363337+00:00 |
| previous_ti | <TaskInstance: nd201905_kpi.getCNCEData 2025-12-16 04:52:25.593744+00:00 [success]> |
| previous_ti_success | <TaskInstance: nd201905_kpi.getCNCEData 2025-12-16 04:52:25.593744+00:00 [success]> |
| priority_weight | 2 |
| queue | default |
| queued_dttm | 2025-12-16 15:08:05.432345+00:00 |
| raw | False |
| run_as_user | None |
| start_date | 2025-12-16 15:08:09.543839+00:00 |
| state | success |
| task | <Task(PythonOperator): getCNCEData> |
| task_id | getCNCEData |
| test_mode | False |
| try_number | 2 |
| unixname | airflow |
| Attribute | Value |
|---|---|
| dag | <DAG: nd201905_kpi> |
| dag_id | nd201905_kpi |
| depends_on_past | False |
| deps | {<TIDep(Trigger Rule)>, <TIDep(Previous Dagrun State)>, <TIDep(Not In Retry Period)>} |
| do_xcom_push | True |
| downstream_list | [<Task(PythonOperator): writeKpiToPostgres>] |
| downstream_task_ids | {'writeKpiToPostgres'} |
| None | |
| email_on_failure | True |
| email_on_retry | True |
| end_date | None |
| execution_timeout | None |
| executor_config | {} |
| extra_links | [] |
| global_operator_extra_link_dict | {} |
| inlets | [] |
| lineage_data | None |
| log | <Logger airflow.task.operators (INFO)> |
| logger | <Logger airflow.task.operators (INFO)> |
| max_retry_delay | None |
| on_failure_callback | None |
| on_retry_callback | None |
| on_success_callback | None |
| op_args | [] |
| op_kwargs | {} |
| operator_extra_link_dict | {} |
| operator_extra_links | () |
| outlets | [] |
| owner | airflow |
| params | {} |
| pool | default_pool |
| priority_weight | 1 |
| priority_weight_total | 2 |
| provide_context | True |
| queue | default |
| resources | None |
| retries | 1 |
| retry_delay | 0:05:00 |
| retry_exponential_backoff | False |
| run_as_user | None |
| schedule_interval | 0 7,15 * * * |
| shallow_copy_attrs | ('python_callable', 'op_kwargs') |
| sla | None |
| start_date | 2023-01-17T00:00:00+00:00 |
| subdag | None |
| task_concurrency | None |
| task_id | getCNCEData |
| task_type | PythonOperator |
| template_ext | [] |
| template_fields | ('templates_dict', 'op_args', 'op_kwargs') |
| templates_dict | None |
| trigger_rule | all_success |
| ui_color | #ffefeb |
| ui_fgcolor | #000 |
| upstream_list | [<Task(PythonOperator): getCEImplementationData>] |
| upstream_task_ids | {'getCEImplementationData'} |
| wait_for_downstream | False |
| weight_rule | downstream |