java.lang.NullPointerException Ignoring exception during close for [email protected]

1.问题描述：在调试mapreduce辅助排序（二次排序）的过程中，运行程序总是报错

Ignoring exception during close for [email protected]
java.lang.NullPointerException
at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:157)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1269)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:74)
at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:63)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1602)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1491)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:723)
at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:2021)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
8860 [Thread-6] INFO org.apache.hadoop.mapred.LocalJobRunner - map task executor complete.
8872 [Thread-6] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local1887386767_0001
java.lang.Exception: java.lang.NullPointerException
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:157)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1269)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:74)
at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:63)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1602)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1491)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:723)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

2.原因分析：

（1）初步判断是是调用了空指针；

（2）当点击WritableComparator.java:157 会跳转到报错的地方是this.buffer.reset(b1, s1, l1);

public int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) {    try {        this.buffer.reset(b1, s1, l1);        this.key1.readFields(this.buffer);        this.buffer.reset(b2, s2, l2);        this.key2.readFields(this.buffer);        this.buffer.reset((byte[])null, 0, 0);    } catch (IOException var8) {        throw new RuntimeException(var8);    }（3）也就是说WritableComparator buffer没有初始化

public class WritableComparator implements RawComparator, Configurable {    private static final ConcurrentHashMap<Class, WritableComparator> comparators = new ConcurrentHashMap();    private Configuration conf;    private final Class<? extends WritableComparable> keyClass;    private final WritableComparable key1;    private final WritableComparable key2;    private final DataInputBuffer buffer;

（4）查找程序中使用到WritableComparator 相关的类的定义是分组类ReducerGroupingComparator  和第一字段的比较类FirstSecondComparator 没有默认的构造函数

public static class ReducerGroupingComparator  extends WritableComparator{//一定要加默认构造函数，且一定要加super,否则会报异常java.lang.NullPointerException Ignoring exception during close for [email protected]    public ReducerGroupingComparator()    {        super(IntPair.class,true);    }    @Override    public int compare(WritableComparable a, WritableComparable b) {        IntPair pairA=(IntPair)a;        IntPair pairB=(IntPair)b;        //年份相同，返回值为0的分为一组        return (int) (pairA.getFirst()-pairB.getFirst());    }}//与FirstSecondComparator中定义的默认对比函数功能相同，这个用于显示设置对比类public  static class FirstSecondComparator extends WritableComparator{    //

//一定要加默认构造函数，且一定要加super,否则会报异常java.lang.NullPointerException Ignoring exception during close for [email protected]

    public FirstSecondComparator()    {        super(IntPair.class,true);    }    @Override    public int compare(WritableComparable a, WritableComparable b) {        IntPair pairA=(IntPair)a;        IntPair pairB=(IntPair)b;        int cmp= (int) (pairA.getFirst()-pairA.getFirst());        if (cmp!=0)        {            return cmp;        }        double A=pairA.getSecond();        double B=pairB.getSecond();        return -((A==B)? 0:(A>B? 1:-1));    }}（5）为什么一定要加这个才不报java.lang.NullPointerException异常？点开super(IntPair.class,true);

会跳转到WritableComparator的构造函数

protected WritableComparator(Class<? extends WritableComparable> keyClass, boolean createInstances) {    this(keyClass, (Configuration)null, createInstances);}在点击this，进一步跳转到另外一个重载构造函数，看到这里大家应该名明白了，上面报异常的地方，就是这些变量没有申请内存空间

protected WritableComparator(Class<? extends WritableComparable> keyClass, Configuration conf, boolean createInstances) {    this.keyClass = keyClass;    this.conf = conf != null ? conf : new Configuration();    if (createInstances) {        this.key1 = this.newKey();        this.key2 = this.newKey();        this.buffer = new DataInputBuffer();    } else {        this.key1 = this.key2 = null;        this.buffer = null;    }

}

自己开发了一个股票智能分析软件，功能很强大，需要的点击下面的链接获取：

https://www.cnblogs.com/bclshuai/p/11380657.html

原文地址：https://www.cnblogs.com/bclshuai/p/12329390.html