JSON and JSONL: Objects, Arrays, and Streaming Records

Hyacehila
  2025-10-14 15:54:21 2025-10-14 15:54:21 Created      

What's JSON?

JS = JavaScript; ON =Object Notation, so JSON means the JavaScript object symbol system, but he already has almost all language versions that are used in any time that both machines and humans are required to read twice..json It means as a suffix.

The questions in this article can also be addressedYAML Format and Usage Speed、SQL Foundation: Query, Aggregation, JOIN and Window FunctionsHow the concept of a relatively close read together is developed in different contexts.

JSON uses {}As a core object, a symbol, and using all key values for tissue data, we use commas to separate the columns within an object, except for the last.

{
"name" : "Jake", //这是一个普通字符串
"age" : 25,  //这是一个数字
"hobbies":["Swimming","Basketball"] //这是一个数组结构
}

JSON allows nesting, which allows you to embed other objects in a JSON object value, which allows us to use multilayered nesting to manage and use complex data.

Except for this use.{}Aside from the structure, JSON has a special object array structure designed to store a large number of duplicate entries in similar formats, which he uses directly in JSON files. [] Store objects in a single array, which are often in the form of JSON because of the embedded devices that support nature.

[
  {
    "id": 1,
    "text": "这是我的第一篇博文，内容很精彩。",
    "images": [
      "/posts/2023/10/20/img1.jpg",
      "/posts/2023/10/20/img2.jpg"
    ],
    "timestamp": "2023-10-20T10:00:00Z",
    "location": "北京"
  },
  {
    "id": 2,
    "text": "今天去了长城，风景真不错。",
    "images": [
      "/posts/2023/10/21/great_wall_a.jpg",
      "/posts/2023/10/21/great_wall_b.jpg",
      "/posts/2023/10/21/great_wall_c.jpg"
    ],
    "timestamp": "2023-10-21T15:30:00Z",
    "location": "北京, 八达岭长城"
  },
  // ... 省略 9997 条
]

JSONL File

When a single JSON file (usually a JSON file for object arrays) becomes very large (e.g. hundreds of MB or several GBs), it is possible to use JSON Lines format when the memory is unrealistic. .jsonl He's used to deal with questions like the object array.

.jsonl Each line of the file is a separate, complete JSON object, with no commas between the rows and the line, and the entire file is not squared. []. Note that the JSON objects in the JSONL file are not embedded in the system, and we don't need him to handle the multi-layer resolution.

{"id": 1, "text": "这是我的第一篇博文，内容很精彩。", "images": ["/posts/2023/10/20/img1.jpg", "/posts/2023/10/20/img2.jpg"], "timestamp": "2023-10-20T10:00:00Z", "location": "北京"}


{"id": 2, "text": "今天去了长城，风景真不错。", "images": ["/posts/2023/10/21/great_wall_a.jpg", "/posts/2023/10/21/great_wall_b.jpg", "/posts/2023/10/21/great_wall_c.jpg"], "timestamp": "2023-10-21T15:30:00Z", "location": "北京, 八达岭长城"}


{"id": 3, "text": "分享一个技术心得…", "images": [], "timestamp": "2023-10-22T09:00:00Z", "location": "上海"}
// … 每行一篇博文

.jsonl Format supportFluid processing, read and processed by line, without a single load of the entire file to the memory, which is extremely low ... andEasy to addJust add a line to the end.

.jsonl The format is essentially multiple JSON files, which cannot be accessed at random, only from the first line, individually, and cannot be analysed directly using a JSON-like method.

How to deal with JSON

We used JS and Python most often to process JSON data, of whichJSON.parseThis originally supported JS function can handle it as an JS-supported object, and then choose to use square brackets or dots to access the embedded object.

Here's a more detailed explanation of Python's position.jsonThese two very important and easily confused functions in the module:json.loads() and json.load()They converted the JSON files to the Python dictionary, then used the square brackets in the Python dictionary. json.JSONDecodeError

A sentence summing up the core distinction:s Representative string(strings) json.loads() to parse strings, and json.load() For parse files.

json.loads() Will str, bytes or bytearray Type of example, containing JSON documents, inverse sequence to Python objects, generally used for API response processing.

import json
# 其中 s 是我们希望解析的字符串,他符合JSON的结构语言
json.loads(s, *, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)

The following conversion methods are generally followed:

  • object {} -> dict dict
  • array [] -> list list
  • string "" -> str str
  • number (int) -> int int
  • number (real) -> float float
  • true -> True
  • false -> False
  • null -> None

load It's a function of one.File-Category Object. Here's the file class object' for any support .read() The most common object is the person who passed the method. open() function opens the file.

import json
# 其中 fp 是我们希望解析的文件,他符合JSON的结构语言
json.load(fp, *, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)

Use the code below, you need to open the file first, then use it json.load Processing

import json


文件路径
file_path = 'data.json'


使用 'with open' 语句打开文件，这是最佳实践
它能确保文件在操作完成后被自动关闭，即使发生错误也不例外
try:
    with open(file_path, 'r', encoding='utf-8') as f:
        # 使用 json.load() 从文件对象 f 中解析JSON数据
        python_data = json.load(f)


    # 查看解析后的Python对象及其类型
    print("从文件解析后的Python对象:")
    print(python_data)
    print(f"类型: {type(python_data)}")

    # 操作数据
    print(f"用户邮箱: {python_data['email']}")
    print(f"标签数量: {len(python_data['tags'])}")



except FileNotFoundError:
    print(f"错误：文件 {file_path} 未找到。")
except json.JSONDecodeError as e:
    print(f"JSON解析错误: {e}")

How to deal with JSONL

First of all, let's be clear. The whole JSONL document itself.No, it's not.A valid JSON object or array. Not available json.load() Read the whole document once, because it's wrong, saying it's not in the right format.Read by line and use each line json.loads().

Assuming there's a JSONL file.

# data.jsonl
{"name": "Alice", "age": 30, "city": "New York"}
{"name": "Bob", "age": 25, "city": "Los Angeles"}
{"name": "Charlie", "age": 35, "city": "Chicago"}

Read it with Python, of course, but it's just one example, not enough for Robustness:

import json


存储解析后的数据
data_list = []


使用 'with' 语句可以确保文件被正确关闭
指定 encoding='utf-8' 是一个好习惯，可以避免编码问题
with open('data.jsonl', 'r', encoding='utf-8') as f:
    for line in f:
        # json.loads() 将每一行的字符串转换为 Python 字典
        data = json.loads(line)
        data_list.append(data)
        print(f"读取到一行数据: {data}")


print("\n所有数据已加载到列表中:")
print(data_list)

Like JSONL itself, we finally got a list of dictionarys in Python.

Write JSON.

Write Standard.json , use this method when you have a complete data structure (e.g. a list of all the Bodicals) and you want to store it as a single, formatted JSON file.

Key Functions: json.dump(data, file_object)

  • data: Python objects (dictionaries, lists, etc.) that you want to write
  • file_object: An open file object
import json


我们的Python数据结构：一个包含多个字典的列表
blog_posts = [
    {
        "id": 1,
        "text": "今天天气真好",
        "images": ["sunny.jpg"],
        "timestamp": "2023-10-20T10:00:00Z"
    },
    {
        "id": 2,
        "text": "分享一篇好文章",
        "images": [],
        "timestamp": "2023-10-20T11:30:00Z"
    },
    {
        "id": 3,
        "text": "我的新宠物",
        "images": ["cat1.jpg", "cat2.jpg"],
        "timestamp": "2023-10-20T12:15:00Z"
    }
]


将数据写入 posts.json 文件
with open('posts.json', 'w', encoding='utf-8') as f:
    # 使用 json.dump() 写入数据
    # indent=4 使文件格式化，易于阅读
    # ensure_ascii=False 确保中文字符能正常写入，而不是被转义
    json.dump(blog_posts, f, indent=4, ensure_ascii=False)


print("成功写入 posts.json 文件")

Result generated

[
    {
        "id": 1,
        "text": "今天天气真好",
        "images": [
            "sunny.jpg"
        ],
        "timestamp": "2023-10-20T10:00:00Z"
    },
    {
        "id": 2,
        "text": "分享一篇好文章",
        "images": [],
        "timestamp": "2023-10-20T11:30:00Z"
    },
    {
        "id": 3,
        "text": "我的新宠物",
        "images": [
            "cat1.jpg",
            "cat2.jpg"
        ],
        "timestamp": "2023-10-20T12:15:00Z"
    }
]

Write.jsonl When it takes a flow of data, or when it's very large, and it doesn't want to be loaded into memory once, use JSONL. Each line is an independent JSON object.

Key Functions: json.dumps(data)

  • data: Python objects you want to convert (usually a single dictionary)
  • This function returns a string in JSON format
import json


相同的博文数据
blog_posts = [
    {
        "id": 1,
        "text": "今天天气真好",
        "images": ["sunny.jpg"],
        "timestamp": "2023-10-20T10:00:00Z"
    },
    {
        "id": 2,
        "text": "分享一篇好文章",
        "images": [],
        "timestamp": "2023-10-20T11:30:00Z"
    },
    {
        "id": 3,
        "text": "我的新宠物",
        "images": ["cat1.jpg", "cat2.jpg"],
        "timestamp": "2023-10-20T12:15:00Z"
    }
]


将数据逐行写入 posts.jsonl 文件
with open('posts.jsonl', 'w', encoding='utf-8') as f:
    # 遍历列表中的每一个字典（博文）
    for post in blog_posts:
        # 1. 使用 json.dumps() 将单个字典转换为JSON字符串
        json_string = json.dumps(post, ensure_ascii=False)


    # 2. 将字符串写入文件，并在末尾添加一个换行符 \n
    f.write(json_string + '\n')



print("成功写入 posts.jsonl 文件")

Result generated

{"id": 1, "text": "今天天气真好", "images": ["sunny.jpg"], "timestamp": "2023-10-20T10:00:00Z"}
{"id": 2, "text": "分享一篇好文章", "images": [], "timestamp": "2023-10-20T11:30:00Z"}
{"id": 3, "text": "我的新宠物", "images": ["cat1.jpg", "cat2.jpg"], "timestamp": "2023-10-20T12:15:00Z"}
  • Title: JSON and JSONL: Objects, Arrays, and Streaming Records
  • Author: Hyacehila
  • Created at : 2025-10-14 15:54:21
  • Link: https://hyacehila.github.io//blog/2025/10/14/json-and-jsonl-learning-notes/
  • License: This work is licensed under CC BY-NC-SA 4.0.
Comments
On this page
JSON and JSONL: Objects, Arrays, and Streaming Records
  1. What's JSON?
  2. JSONL File
  3. How to deal with JSON
  4. 文件路径
  5. 使用 'with open' 语句打开文件，这是最佳实践
  6. 它能确保文件在操作完成后被自动关闭，即使发生错误也不例外
    1. How to deal with JSONL
  7. 存储解析后的数据
  8. 使用 'with' 语句可以确保文件被正确关闭
  9. 指定 encoding='utf-8' 是一个好习惯，可以避免编码问题
    1. Write JSON.
  10. 我们的Python数据结构：一个包含多个字典的列表
  11. 将数据写入 posts.json 文件
  12. 相同的博文数据
  13. 将数据逐行写入 posts.jsonl 文件