java.nio.charset.MalformedInputException: Input length = 1

2017-08-22 17:11:11   作者:MangoCool   来源:MangoCool

 flume流化文件报错:java.nio.charset.MalformedInputException: Input length = 1,直接挂掉。

详细信息:

2017-08-22 16:49:42,819 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r_ggsn: { spoolDir: /data1/kafka/data2/ggsndata_flume }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.
java.nio.charset.MalformedInputException: Input length = 1
	at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
	at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)
	at org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:133)
	at org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:71)
	at org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:90)
	at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:252)
	at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:228)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

原因:flume默认读取utf-8编码的文件,但是文件本身不是utf-8编码。

解决办法:

iconv filename.txt -f ISO-8859-1 -t UTF-8 -o filename_2.txt --verbose
这里假设文件编码为ISO-8859-1。

标签: flume 流化 编码

分享:

关于我

一个喜欢唱歌,热衷旅行,爱好电子产品的码农。没事,跟三五好友吼上几嗓子,约上几个背着行囊去露营或者宅在家里抱着孩子敲代码。

座右铭:当你的才华还撑不起你的野心的时候,你就应该静下心来学习,永不止步!

            人生之旅历途甚长,所争决不在一年半月,万不可因此着急失望,招精神之萎葸。


Copyright 芒果酷(mangocool.com) All rights reserved. 湘ICP备14019394号

免责声明:本网站部分文章转载其他媒体,意在为公众提供免费服务。如有信息侵犯了您的权益,可与本网站联系,本网站将尽快予以撤除。