解决kylin报错：Failed to create dictionary on <db>.<table>, Caused by: java.lang.IllegalArgumentException: Too high cardinality is not suitable for dictionary

报错信息：

2017-05-13 15:14:30,035 DEBUG [pool-9-thread-10] dict.DictionaryGenerator:94 : Dictionary class: org.apache.kylin.dict.TrieDictionary
2017-05-13 15:14:30,036 ERROR [pool-9-thread-10] common.HadoopShellExecutable:65 : error execute HadoopShellExecutable{id=2657ff38-35b0-4a33-9f6b-fee48031147f-03, name=B
uild Dimension Dictionary, state=RUNNING}
java.lang.RuntimeException: Failed to create dictionary on HM_30_1_PROD_20170418_20170512.OUTPATIENT_VISIT_ANTIBIOTICS.VISIT_SN
        at org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:325)
        at org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:222)
        at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:50)
        at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:41)
        at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:54)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
        at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
        at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Too high cardinality is not suitable for dictionary -- cardinality: 5980217
        at org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:96)
        at org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryGenerator.java:73)
        at org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:321)
        ... 14 more

解决方案：

1. 请使用其他编码方式，例如“fixed_length”，“integer”等。

2. Apache Kylin中对上亿字符串的精确Count_Distinct示例 – lxw的大数据田地

时间： 2024-10-25 09:23:23

解决kylin报错：Failed to create dictionary on <db>.<table>, Caused by: java.lang.IllegalArgumentException: Too high cardinality is not suitable for dictionary的相关文章

Linux下启动tomcat报错RROR org.apache.catalina.core.StandardContext- Error starting static Resources java.lang.IllegalArgumentException: Document base /home/duiba/apache-tomcat/webapps/../webapps/manager do

部署项目的时候,重启tomcat,死活起不来,很郁闷,网上巴拉了半天,结合自己的情况,找到了原因: 错误日志信息: 2018-12-13 13:52:26,992 [main] INFO org.apache.catalina.core.AprLifecycleListener- The APR based Apache Tomcat Native library which allows optimal performance in production environments was no

MyEclipse打开 HTML 报错Failed to create the part&#39;s controls

拷贝代码时有时会弹出这个错误,页面仍然可以访问,但是无法编辑很郁闷. MyEclipse默认打开编辑页面是MyEclipse visual html designer 右击html页面选择open with,点击 MyEclipse html Editer即可. MyEclipse打开 HTML 报错Failed to create the part's controls

Mybatis 报错 There is no getter for property named &#39;***&#39; in &#39;class java.lang.String&#39;

在mapper.xml中 , 如果单参数是String类型 , 且在sql语句中对参数进行了判断 , 如下 when 中的判断 , 如果出现 if 判断也是一样的.都需要把判断中的参数用 _parameter 来代替 ,. 另外orcal中判断字段是否为空需要使用 is null , 同理,判断不为空使用 is not null . 错误查询: <select id = "select" resultMap="ResultMap" parameterType=

使用mybatis报错constructor found in com.atguigu.mybatis.bean.Department matching [java.lang.Integer, java.lang.String]

报错constructor found in com.atguigu.mybatis.bean.Department matching [java.lang.Integer, java.lang.String] 构造方法不匹配,因为反射中会使用默认的无参构造器,而当我们一旦写了新的带参构造方法,那么就不会存在无参构造方法了,需要重新写过.自己在相应的实体中写一个无参构造方法就可以了. 完整报错如下所示: org.apache.ibatis.exceptions.PersistenceExcept

解决Caused by: java.lang.IllegalArgumentException: Result Maps collection does not contain value for com.geek.dao.ContentDao.Integer

mybatis报错:Caused by: java.lang.IllegalArgumentException: Result Maps collection does not contain value for com.geek.dao.ContentDao.Integer 出错原因:一个手误 <select id="selectAllFirstNavigationName" resultMap="firstNavigation"> SELECT fi

启动Myeclipse报错“Failed to create the Java Virtual Machine”的解决办法

我安装的是Myeclipse 10.7.1.装上好久没用,今天启动突然报错:Failed to create the Java Virtual Machine. 检查Myeclipse安装好使用时好的啊,近期也没用,可能是近期升级了本地单独安装的jre版本导致的吧(Myeclipse使用自己的jre的). 整理了如下2个解决办法,可以选择一个使用,我选择的是第2个.经测试都ok. 方法一: 找到Myeclpise路径下的myeclipse.ini文件: 编辑将Xmx(JVM Heap最大允许的尺

解决Eclipse启动报错Failed to create the Java Virtual Machine

电脑:2G内存,WIN7 32位. 启动adt-bundle-windows-x86-20140702\eclipse\eclipse.exe时,报错[Failed to create the Java Virtual Machine].点确定后,Eclipse自动关闭 解决方法如下: 修改Eclipse目录下的配置文件[eclipse.iniadt-bundle-windows-x86-20140702\eclipse\eclipse.ini] 将下面两个值改为原来的0.5倍,然后保存,重新启

Eclipse及Eclipse为基础的App报错“Failed to create the Java Virtual Machine”的解决办法

由于OracleJDK马上就要收费了,公司要求更换OpenJDK,结果安装后Eclipse及Eclipse为基础的App启动报错:"Failed to create the Java Virtual Machine" 根据理解和经验此文特地整理了有关修复方法. 方法一:找到eclpise路径下的eclipse.ini文件,编辑将Xmx(JVM Heap最大允许的尺寸)修改为原值一半大小.比如原值为512,则修改为256. 此方法是比较老的方法,也是目前网上流行的方法.一些新的版本可能不

IDEA 运行报错 failed to create a child event loop

背景 在IDEA中写了测试代码,但是运行的时候一直提示 java.lang.IllegalStateException: failed to create a child event loop ... eclipse中同样的代码运行没有问题,于是搜索了一些关于这个错误的解决方法,基本上说是防火墙引起的,需要在防火墙允许应用运行中添加例外应用,添加IDEA即可. 但是,尝试了以后,还是不行...... Error:Abnormal build process termination: C:\Con