解除HUE下载10万行的限制

绝地灬酷狼 2023-07-24 02:58 25阅读 0赞

在hue里对查询的数据导出到本地的时候只能下载10万条记录，这个局限性比较大，影响到日常的使用，通过查看源代码可以看到hue是对单元格以及列数量进行了限制：1000万单元格/100列=10万行数据

到hue的hue server实例机器上找到源代码:

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dzZGMwNTIx_size_16_color_FFFFFF_t_70][]

**vim /opt/cloudera/parcels/CDH/lib/hue/apps/beeswax/src/beeswax/conf.py**

# Deprecated
    DOWNLOAD_CELL_LIMIT = Config(
      key='download_cell_limit',
      default=10000000,    #这里限制了1000万单元格上限
      type=int,
      help=_t('A limit to the number of cells (rows * columns) that can be downloaded from a query '
              '(e.g. - 10K rows * 1K columns = 10M cells.) '
              'A value of -1 means there will be no limit.'))
    
    def get_deprecated_download_cell_limit(): #此函数使用上面函数的1000万/100=10万行返回
      """Get the old default"""
      return DOWNLOAD_CELL_LIMIT.get() / 100 if DOWNLOAD_CELL_LIMIT.get() > 0 else DOWNLOAD_CELL_LIMIT.get()
    
    DOWNLOAD_ROW_LIMIT = Config(
      key='download_row_limit',
      dynamic_default=get_deprecated_download_cell_limit,  #此参数为行数限制
      type=int,
      help=_t('A limit to the number of rows that can be downloaded from a query before it is truncated. '
              'A value of -1 means there will be no limit.'))

通过help里的提醒可以发现，只要修改default=-1即可同时解除行列的下载限制。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dzZGMwNTIx_size_16_color_FFFFFF_t_70 1][]

修改之后保存重启hue服务即可解除下载行数限制:

# Deprecated
    DOWNLOAD_CELL_LIMIT = Config(
      key='download_cell_limit',
      #default=10000000,
      default=-1,  #改这里即可
      type=int,
      help=_t('A limit to the number of cells (rows * columns) that can be downloaded from a query '
              '(e.g. - 10K rows * 1K columns = 10M cells.) '
              'A value of -1 means there will be no limit.'))
    
    def get_deprecated_download_cell_limit():
      """Get the old default"""
      return DOWNLOAD_CELL_LIMIT.get() / 100 if DOWNLOAD_CELL_LIMIT.get() > 0 else DOWNLOAD_CELL_LIMIT.get()
    
    DOWNLOAD_ROW_LIMIT = Config(
      key='download_row_limit',
      dynamic_default=get_deprecated_download_cell_limit,
      type=int,
      help=_t('A limit to the number of rows that can be downloaded from a query before it is truncated. '
              'A value of -1 means there will be no limit.'))

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dzZGMwNTIx_size_16_color_FFFFFF_t_70]: /images/20230528/418f7d7e0e214af887eebf4eabc3ca29.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dzZGMwNTIx_size_16_color_FFFFFF_t_70 1]: /images/20230528/2625a365cea64f0a82add49ce191aa45.png