利用ctc训练字符识别

亦凉 2023-06-29 02:49 17阅读 0赞

### 一、获取字体 ###

*  win10字体查找方法  **\[Windows + E\] -> **%**WINDIR%/Fonts**
 *  CentOS Linux** **字体路径： **/usr/share/fonts**

root@9080e45b4485:~# apt-get install  fontconfig
    root@9080e45b4485:~# fc-list
    /usr/share/fonts/truetype/dejavu/DejaVuSerif-Bold.ttf: DejaVu Serif:style=Bold
    /usr/share/fonts/truetype/dejavu/DejaVuSansMono.ttf: DejaVu Sans Mono:style=Book
    /usr/share/fonts/truetype/dejavu/DejaVuSans.ttf: DejaVu Sans:style=Book
    /usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf: DejaVu Sans:style=Bold
    /usr/share/fonts/truetype/dejavu/DejaVuSansMono-Bold.ttf: DejaVu Sans Mono:style=Bold
    /usr/share/fonts/truetype/dejavu/DejaVuSerif.ttf: DejaVu Serif:style=Book

*  **matplotlib字体路径 /usr/local/lib/python3.5/dist-packages/matplotlib/mpl-data/fonts/ttf/**

### 二、生成文件 ###

*  利用字库文件和文字内容生成训练数据

class MYFONT():
    	def __init__(self, font_path='data'):
    		self._Deng_path = 'Deng.ttf'
    		self.__load_font(font_path)
    
    	def __load_font(self, font_path):
    		self.Deng_font = ImageFont.truetype(os.path.join(font_path, self._Deng_path), 23, encoding="unic")
    
    
    	def set_font_size(self, font_size=71):
    	    txt_font_tmp = txt_font.font_variant(size=20, encoding='unic')
    		return txt_font_tmp
    
    	def get_txt_cord(self, txt, font):  # 字体的坐标信息：[x,y,w,h]
    		offset = font.getoffset(txt)
    		size = font.getsize(txt)
    		return offset + size  # x,y,w,h
    
    	def get_alpha_new_image(self, imgwh):
    		img_w, img_h = imgwh
    
    		pil_img = Image.new('RGB', (img_w, img_h), (255, 255, 255))
    		return pil_img
    
    	def get_txt_part_base(self, imgwh, txt, txt_cord, txt_font):
    		'''
    		将文字写在图片上，白底黑字
    		:param imgwh: 目标图分辨率
    		:param txt: 待写的文字list变量
    		:param txt_cord: 对应坐标
    		:param txt_font: 对应字体
    		:return: 结果图
    		'''
    		pil_img = self.get_alpha_new_image(imgwh)
    		pil_img_draw = ImageDraw.Draw(pil_img)
    
    
    		pil_img_draw.text(txt_cord, txt, (0, 0 , 0), font=txt_font)
    		return pil_img

*  [生成lmdb数据][lmdb]

#encoding: utf-8
    import os
    import lmdb # install lmdb by "pip install lmdb"
    import cv2
    import numpy as np
    
    '''
    来自https://github.com/bgshih/crnn/blob/master/tool/create_dataset.py
    '''
    
    def checkImageIsValid(imageBin):
        if imageBin is None:
            return False
        imageBuf = np.fromstring(imageBin, dtype=np.uint8)
        img = cv2.imdecode(imageBuf, cv2.IMREAD_GRAYSCALE)
        imgH, imgW = img.shape[0], img.shape[1]
        if imgH * imgW == 0:
            return False
        return True
    
    
    def writeCache(env, cache):
        with env.begin(write=True) as txn:
            #for k, v in cache.iteritems(): #py2
            for k, v in cache.items(): #py3
                #txn.put(k, v) #py2
                txn.put(str(k).encode(), v) #py3
    
    
    def createDataset(outputPath, imagePathList, labelList, lexiconList=None, checkValid=True):
        """
        Create LMDB dataset for CRNN training.
    
        ARGS:
            outputPath    : LMDB output path
            imagePathList : list of image path
            labelList     : list of corresponding groundtruth texts
            lexiconList   : (optional) list of lexicon lists
            checkValid    : if true, check the validity of every image
        """
        assert(len(imagePathList) == len(labelList))
        nSamples = len(imagePathList)
        env = lmdb.open(outputPath, map_size=1099511627776)
        cache = {}
        cnt = 1
        for i in range(nSamples):
            imagePath = imagePathList[i]
            label = labelList[i]
            if not os.path.exists(imagePath):
                print('%s does not exist' % imagePath)
                continue
            #print (imagePath)
            with open(imagePath, 'rb') as f:
                imageBin = f.read()
            if checkValid:
                if not checkImageIsValid(imageBin):
                    print('%s is not a valid image' % imagePath)
                    continue
    
            imageKey = 'image-%09d' % cnt
            labelKey = 'label-%09d' % cnt
            cache[imageKey] = imageBin
            cache[labelKey] = label
            if lexiconList:
                lexiconKey = 'lexicon-%09d' % cnt
                cache[lexiconKey] = ' '.join(lexiconList[i])
            if cnt % 1000 == 0:
                writeCache(env, cache)
                cache = {}
                print('Written %d / %d' % (cnt, nSamples))
            cnt += 1
        nSamples = cnt-1
        # cache['num-samples'] = nSamples #py2
        cache['num-samples'] = str(nSamples).encode() #py3
        writeCache(env, cache)
        print('Created dataset with %d samples' % nSamples)
    
    
    def getAllUrls(txt):
        rtn = list()
        with open(txt, 'rb') as fp:
            buffs = fp.readlines()
        rtn = [tmp.strip() for tmp in buffs]
        return rtn
    
    if __name__ == '__main__':
        img_pathfile = 'imgslist.txt'  #文件列表
        labellist = 'labellist.txt'    #文件对应的文字内容
        outpath = 'trainlmdb'
        if not os.path.exists(outpath):
            os.makedirs(outpath)
        imagePathList = getAllUrls(img_pathfile)
        labelList = getAllUrls(labellist)
        createDataset(outpath, imagePathList, labelList)

### 三、训练crnn ###

*  安装[pytorch ctc-loss][]
 *  训练项目地址[crnn.pytorch][]
 *  注意事项：1.使用py2环境，torch使用1.1.0；2. 学习率需要修改为0.0001，默认参数0.01训练loss无穷大。

问题

*  Corrupted image 657。 表明生成lmdb数据时存在问题，lmdb数据加载过程可参考[这里][Link 1]

#生成ldmb数据后，可在crnn.pytorch根目录中运行该程序测试数据是否正常
    import torch
    import dataset   
    test_dataset = dataset.lmdbDataset(
        root='carplate_lmdb/val/', transform=dataset.resizeNormalize((100, 32)))
    data_loader = torch.utils.data.DataLoader(
            test_dataset, shuffle=True, batch_size=64, num_workers=1)
    val_iter = iter(data_loader)

*  RuntimeError: [set\_sizes\_contiguous is not allowed ][set_sizes_contiguous is not allowed]on Tensor created from .data or .detach()。原因是版本问题，可参考[这里][Link 2]和[这里][set_sizes_contiguous is not allowed]。

修改utils.py中loadData函数体 
    v.data.resize_(data.size()).copy_(data) -> v.resize_(data.size()).copy_(data)

*  [RuntimeError: dimension out of range (expected to be in range of \[-2, 1\], but got 2)][RuntimeError_ dimension out of range _expected to be in range of _-2_ 1_ but got 2]

preds = preds.squeeze(2)  #注释该行

**参考文献：**

1.  [PyTorch bindings for Warp-ctc][pytorch ctc-loss]
2.  [crnn.pytorch][]
3.  [Location of truetype font][]
4.  [车牌及车标查询][Link 3]
5.   [lmdb 模块实例源码][Link 1]

[lmdb]: https://github.com/bgshih/crnn/blob/master/tool/create_dataset.py
[pytorch ctc-loss]: https://github.com/SeanNaren/warp-ctc
[crnn.pytorch]: https://github.com/meijieru/crnn.pytorch
[Link 1]: http://codingdict.com/sources/py/lmdb/18429.html
[set_sizes_contiguous is not allowed]: https://blog.csdn.net/abstract_think/article/details/91410703
[Link 2]: https://github.com/jwyang/faster-rcnn.pytorch/issues/543
[RuntimeError_ dimension out of range _expected to be in range of _-2_ 1_ but got 2]: https://github.com/meijieru/crnn.pytorch/issues/31
[Location of truetype font]: https://superuser.com/questions/137826/location-of-truetype-fonts
[Link 3]: https://www.baichew.com/chebiao/guochan/