Clickhouse 无法正接入接入Kafka数据

发布于 5 个月前 作者 boomballa 1278 次浏览 来自 问答

模拟kafka消费了一条数据,数据情况如下: [songpeng01@hz-forward1 ~]$ cat test.json |python -mjson.tool

{ “contentId”: “undefined”, “contentType”: “other”, “day”: “2021-04-01”, “docId”: false, “domain”: “www.xxx.com”, “pageType”: “index”, “path”: “/”, “referer”: “”, “spiderHost”: “xxxx”, “status”: “200”, “timestamp”: 1617247239000, “url”: “/other”, “userAgent”: “xxxxx/2.1.0+(Linux;+U;+Android+9;+BRAVIA+4K+UR2+Build/PTT1.190515.001.S97)”, “videoId”: false }

Clickhouse建表如下:

CREATE TABLE druid.druid_spider_kfk ( contentId String, contentType String, day Date, docId String, domain String, pageType String, path String, referer String, spiderHost String, status String, timestamp UInt64, url String, userAgent String, videoId String ) ENGINE = Kafka SETTINGS kafka_broker_list = ‘kafka_host1:9092,kafka_host2:9092,kafka_host3:9092’, kafka_topic_list = ‘druid-spider-topic’, kafka_group_name = ‘group-clickhouse’, kafka_format = ‘JSONEachRow’, kafka_num_consumers = 1;

物化视图创建如下: CREATE MATERIALIZED VIEW druid.tab_mv TO druid.druid_spider_tab ( contentId String, contentType String, day Date, docId String, domain String, pageType String, path String, referer String, spiderHost String, status String, timestamp UInt64, url String, userAgent String, videoId String ) AS SELECT contentId, contentType, day, docId, domain, pageType, path, referer, spiderHost, status, timestamp, url, userAgent, videoId FROM druid.druid_spider_kfk;

CREATE TABLE druid.druid_spider_tab ( contentId String, contentType String, day Date, docId String, domain String, pageType String, path String, referer String, spiderHost String, status String, timestamp UInt64, url String, userAgent String, videoId String ) ENGINE = MergeTree() PARTITION BY day ORDER BY day SETTINGS index_granularity = 8192;

select * from druid.druid_spider_tab; 0 rows in set. Elapsed: 0.002 sec.

不知道为什么不能接受数据,不知道该怎么排查,有没有朋友能帮忙解惑一下 ,十分感谢。

回到顶部