开发者

How to properly write regexp_extract in druid ingestion spec?

开发者 https://www.devze.com 2022-12-07 23:35 出处:网络
I am trying to write a transformation spec in druid with regexp_extract. The regexp_extract function w开发者_开发知识库orks fine in the query itself. The column data looks something like

I am trying to write a transformation spec in druid with regexp_extract. The regexp_extract function w开发者_开发知识库orks fine in the query itself. The column data looks something like

{
"ID":"2",....
}
SELECT
regexp_extract(input, '(?<=\"ID\":\")(\d+)(?=\",)', 0) AS ID
FROM data
WHERE __time >= CURRENT_TIMESTAMP - INTERVAL '5' HOUR ORDER BY __time DESC

However, I am having trouble in putting it in ingestion spec under transformation spec. I have something like

     "transformSpec": {
        "filter": null,
        "transforms": [
          {
            "type": "expression",
            "name": "ID",
            "expression": "regexp_extract(input, '(?<=\\\"ID\\\":\\\")(\\d+)(?=\\\",)', 0)"
          }
        ]
      }

2022-12-06T23:38:23,877 INFO [main] org.apache.druid.server.coordination.SegmentLoadDropHandler - Stopped.
2022-12-06T23:38:23,877 INFO [main] org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner - Starting graceful shutdown of task[index_kafka_auditLogs_7e8581e66fdf59a_mlabfboa].
line 1:22 token recognition error at: ''(?<=\"ID\":\")(\d'
line 1:44 no viable alternative at input '+'
line 1:47 token recognition error at: '?'
line 1:48 token recognition error at: '=\'
line 1:50 token recognition error at: '",)', 0)'
line 1:46 extraneous input '(' expecting {<EOF>, '-', '^', '*', '/', '%', '+', '<', '<=', '>', '>=', '==', '!=', '&&', '||'}
2022-12-06T23:38:23,878 ERROR [main] org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler - Exception when stopping method[public void org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner.stop()] on object[org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner@3d41449a]
java.lang.reflect.InvocationTargetException: null
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
    at org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler.stop(Lifecycle.java:465) [druid-core-0.23.0-8.jar:0.23.0-8]
    at org.apache.druid.java.util.common.lifecycle.Lifecycle.stop(Lifecycle.java:368) [druid-core-0.23.0-8.jar:0.23.0-8]
    at org.apache.druid.cli.CliPeon.run(CliPeon.java:325) [druid-services-0.23.0-8.jar:0.23.0-8]
    at org.apache.druid.cli.Main.main(Main.java:114) [druid-services-0.23.0-8.jar:0.23.0-8]
Caused by: java.lang.NullPointerException
    at org.apache.druid.query.expression.ExprUtils.isStringLiteral

This is what the ingestion spec looks like. Can anyone please help, how I can properly specify the regex expression?

0

精彩评论

暂无评论...
验证码 换一张
取 消