spark空指针错误可能的原因之一——df.rdd.toDF

电玩女神 2022-12-13 12:39 40阅读 0赞

这空值动不动就Null point error，是个巨大的坑，昨天调试了一天的程序发现，dataframe转rdd进行类似于filter一些的操作，最后count的时候就会报空指针的错误，但是df不转为rdd而是直接df.filter(func)的话，就可以不报错地完美运行。究其根本...待下回分解。

关于spark/Scala的空值处理：

[https://www.cnblogs.com/wwxbi/p/6011422.html][https_www.cnblogs.com_wwxbi_p_6011422.html]

[https://blog.csdn.net/u012798083/article/details/82849438][https_blog.csdn.net_u012798083_article_details_82849438]

[https://blog.csdn.net/ACyoungniu/article/details/50620392?utm\_medium=distribute.pc\_relevant\_t0.none-task-blog-BlogCommendFromMachineLearnPai2-1.channel\_param&depth\_1-utm\_source=distribute.pc\_relevant\_t0.none-task-blog-BlogCommendFromMachineLearnPai2-1.channel\_param][https_blog.csdn.net_ACyoungniu_article_details_50620392_utm_medium_distribute.pc_relevant_t0.none-task-blog-BlogCommendFromMachineLearnPai2-1.channel_param_depth_1-utm_source_distribute.pc_relevant_t0.none-task-blog-BlogCommendFromMachineLearnPai2-1.channel_param]

[https://www.jianshu.com/p/95896d06a94d][https_www.jianshu.com_p_95896d06a94d]

[https://blog.csdn.net/bowenlaw/article/details/104298923][https_blog.csdn.net_bowenlaw_article_details_104298923]

[https_www.cnblogs.com_wwxbi_p_6011422.html]: https://www.cnblogs.com/wwxbi/p/6011422.html
[https_blog.csdn.net_u012798083_article_details_82849438]: https://blog.csdn.net/u012798083/article/details/82849438
[https_blog.csdn.net_ACyoungniu_article_details_50620392_utm_medium_distribute.pc_relevant_t0.none-task-blog-BlogCommendFromMachineLearnPai2-1.channel_param_depth_1-utm_source_distribute.pc_relevant_t0.none-task-blog-BlogCommendFromMachineLearnPai2-1.channel_param]: https://blog.csdn.net/ACyoungniu/article/details/50620392?utm_medium=distribute.pc_relevant_t0.none-task-blog-BlogCommendFromMachineLearnPai2-1.channel_param&depth_1-utm_source=distribute.pc_relevant_t0.none-task-blog-BlogCommendFromMachineLearnPai2-1.channel_param
[https_www.jianshu.com_p_95896d06a94d]: https://www.jianshu.com/p/95896d06a94d
[https_blog.csdn.net_bowenlaw_article_details_104298923]: https://blog.csdn.net/bowenlaw/article/details/104298923