Chinese character riddle is a riddle game in which the riddle solution is a single Chinese character. It is closely connected with the shape, pronunciation or meaning of Chinese characters. The riddle description (sentence) is usually composed of phrases with rich linguistic phenomena (such as pun, simile, and metaphor), which are associated to different parts (namely radicals) of the solution character. In this paper, we propose a statistical framework to solve and generate Chinese character riddles. Specifically, we learn the alignments and rules to identify the metaphors between phrases in riddles and radicals in characters. Then, in the solving phase, we utilize a dynamic programming method to combine the identified metaphors to obtain candidate solutions. In the riddle generation phase, we use a template-based method and a replacement-based method to obtain candidate riddle descriptions. We then use Ranking SVM to rerank the candidates both in the solving and generation process. Experimental results in the solving task show that the proposed method outperform baseline methods. We also get very promising results in the generation task according to human judges.