Workaround a jython runtime bug by interop with Java java.time

TypeError: unsupported operand type(s) for +: 'java.sql.Timestamp' and 'timedelta'

I was working on a NIFI script recently. As the following code snip shows, I was trying to construct a series of timestamped string with Python’s datetime.

import os.path
from datetime import datetime, time, timedelta
import subprocess

now = datetime.now()

dirs = [ (now + timedelta(milliseconds=(-100 * x))).strftime('%Y%m%d%H/%M/%S/%f')[0:-5] for x in range(0,100)]

fullpaths = [ '/workspace/nifi_output/{}'.format(d) for d in dirs]

existspaths = [fp for fp in fullpaths if os.path.exists(fp)]

flowFile = session.get() 
if (flowFile == None):
    flowFile = session.create()

ep_length = len(existspaths)

if ep_length > 1:
    session.putAttribute(flowFile, 'batch_path', existspaths[1])
    session.transfer(flowFile, REL_SUCCESS)
elif ep_length == 1:
    session.putAttribute(flowFile, 'batch_path', existspaths[0])
    session.transfer(flowFile, REL_SUCCESS)
else:
    session.transfer(flowFile, REL_FAILURE)

While the datetime package works perfect in Python2, It refuse to run in Nifi bundled jython runtime. I got a exception:

TypeError: unsupported operand type(s) for +: 'java.sql.Timestamp' and 'timedelta'

It feels like the underlying jython implementation wrapped java.sql.Timestamp to emulate pyhton datetime package. And the type system was confused during the excution of translated bytecode or JIT code.

WorkAround

The workaround: Avoid use of python datatime package whenever your code is targeting a jython runtime.

import os.path
import subprocess
from java.time import LocalDateTime
from java.time.format import DateTimeFormatterBuilder
from java.time.temporal import ChronoUnit

# We use java LocalDateTime instead of datetime.datetime datetime.timedelta
# becasue of Jython bug 
# 
# TypeError: unsupported operand type(s) for +: 'java.sql.Timestamp' and 'timedelta' in <script> at line number 8
#
# ref: https://community.openhab.org/t/using-python-datetime/114917
# ref: https://issues.streamsets.com/browse/SDC-3429
# 
now = LocalDateTime.now()
formatter = DateTimeFormatterBuilder().appendPattern("yyyyMMddHH/mm/ss/S").toFormatter()

localdatetimes = (now.minus(x*100, ChronoUnit.MILLIS) for x in range(100, 0, -1))
timestamps = (ldt.format(formatter) for ldt in localdatetimes)

fullpaths =  ('/opt/ericsson/workspace/nifi_output/{}'.format(ts) for ts in timestamps)

existspaths = [fp for fp in fullpaths if os.path.exists(fp)]


flowFile = session.get() 
if (flowFile == None):
    flowFile = session.create()

ep_length = len(existspaths)

if ep_length > 1:
    session.putAttribute(flowFile, 'batch_path', existspaths[1])
    session.transfer(flowFile, REL_SUCCESS)
elif ep_length == 1:
    session.putAttribute(flowFile, 'batch_path', existspaths[0])
    session.transfer(flowFile, REL_SUCCESS)
else:
    session.transfer(flowFile, REL_FAILURE)

Note that, the DateTimeFormatterBuilder().appendPattern("yyyyMMddHH/mm/ss/S").toFormatter() may be simplified to DateTimeFormatter.ofPattern("yyyyMMddHH/mm/ss/S")

References

• https://community.openhab.org/t/using-python-datetime/114917
• https://issues.streamsets.com/browse/SDC-3429
• https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html