43

spark 存储json数据遇到问题——json4s使用（一）

 5 years ago
source link: https://blog.csdn.net/wild46cat/article/details/54171973?amp%3Butm_medium=referral
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

spark 存储json数据遇到问题——json4s使用（一）

在进行spark的学习过程中遇到了这样的问题。

1、《O'REILLY Learning Spark》的书中说道能够使用spark进行json 的读写，但是给出的例子使用的json转换的jar包是jackson的，还需要再次进行引入。

2、能不能找到一个spark内部已经有的基于scala的json处理的jar包。

在查看spark依赖的jar文件时，找到了json4s这个jar包，很明显就是使用scala的一个json转换的jar包。下面就介绍一下如何使用，首先介绍的是如何通过json4s实现生成json，后面还会介绍如何通过json串转换成对象。

好，下面上货，主要参考的是json4s的官方文档，这是地址：https://github.com/json4s/json4s/。 

import org.json4s._
  import org.json4s.JsonDSL._
  import org.json4s.jackson.JsonMethods._
  
  object JsonStudy {
    def main(args: Array[String]): Unit = {
      val a = parse(""" { "numbers" : [1, 2, 3, 4] } """)
      println(a.toString)
      val b = parse("""{"name":"Toy","price":35.35}""", useBigDecimalForDouble = true)
      println(b.toString)
  
      val c = List(1, 2, 3)
      val d = compact(render(c))
      println(d)
      val e = ("name" -> "joe")
      val f = compact((render(e)))
      println(f)
      val g = ("name" -> "joe") ~ ("age" -> 35)
      val h = compact(render(g))
      println(h)
      val i = ("name" -> "joe") ~ ("age" -> Some(35))
      val j = compact(render(i))
      println(j)
      val k = ("name" -> "joe") ~ ("age" -> (None: Option[Int]))
      val l = compact(render(k))
      println(l)
  
      //定义json
      println("===========================")
      //推荐这种方式，因为可以用在使用map
      val jsonobj = (
        ("name" -> "xiaoming") ~
          ("age" -> 12)
        )
      println(jsonobj)
      println(compact(render(jsonobj)))
  
      val jsonobjp = parse(
        """{
            "name":"xiaogang",
            "age":12
          }""")
      println(jsonobjp)
      println(compact(render(jsonobjp)))
  
      //通过类生成json
      println("===========================")
      case class Winner(id: Long, numbers: List[Int])
      case class Lotto(id: Long, winningNumbers: List[Int], winners: List[Winner], drawDate: Option[java.util.Date])
      val winners = List(Winner(23, List(2, 45, 34, 23, 3, 5)), Winner(54, List(52, 3, 12, 11, 18, 22)))
      val lotto = Lotto(5, List(2, 45, 34, 23, 7, 5, 3), winners, None)
      val json =
        ("lotto" ->
          ("lotto-id" -> lotto.id) ~
            ("winning-numbers" -> lotto.winningNumbers) ~
            ("draw-date" -> lotto.drawDate.map(_.toString)) ~
            ("winners" ->
              lotto.winners.map { w =>
                (("winner-id" -> w.id) ~
                  ("numbers" -> w.numbers))
              }))
  
      println(compact(render(json)))
    }
  }
运行结果：

3m2IRbi.png!web

这里转换的过程已经很清楚了，不再做过多的赘述。


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK