我是如何以 iTunes 为中心管理和全平台(Windows/Mac/Linux/Android/Chromebook/iOS/iPod)同步音乐的

前言


经过多年的探索,我总结了一套最适合我的听音乐和管理音乐的方法,这个方法并不是十分容易实现,并且为了它我也写了一些脚本,因此我把它分享出来,大家如果有需要也可以试试看。

这个方法适合谁

  1. 喜欢收藏经典音乐,喜欢将音乐下载到本地音乐库中听
  2. 有 Windows/Mac,但也希望 Linux/iPod/Android 可以用(有全平台需求)

2019.10.29 更新:解决了itunes2rhythmbox中日韩歌曲无法匹配的问题

这个问题困扰我好久,歌曲名本身看起来是一样的,但是却找不到相应的文件,原因是unicode编码方式会使有的字看起来一样编码却不一样。比如日语中的が本应是一个字符,从mac复制过来后文件名变成了2个字符か和右上角的两个点的两个字符,韩语也同理。我也没有办法穷举出所有的变化,所以就使用源文件与目标文件路径字符串之间的编辑距离以及路径中ascii码的编辑距离两个特征进行匹配,寻找最接近的音乐文件以匹配。新版的itunes2rhythmbox脚本见这个链接

#!/usr/bin/env fish
#!/usr/bin/env python3
import heapq
import multiprocessing
import os
import re
import sys
from itertools import zip_longest
from xml.etree import ElementTree

help_message = '''Usage: Close Rhythmbox, then type: itunes2rhythmbox

If you are to move or sync your whole iTunes library to Rhythmbox, This script
will help you to convert iTunes playlist to Rhythmbox style playlist, and
import the playlist to Rhythmbox directly.

Before using this script, you should have:
- an iTunes folder copied directly from your Windows/Mac computer(with the
  iTunes files already consolidated: in iTunes, click File -> Library ->
  Organize Library -> Consolidate files -> OK), or just the `iTunes Media`
  Folder located at `iTunes/`
- a copy of `iTunes Music Library.xml`, which usually located at `iTunes/`

You should modify this script configurations at start of the script, and the
meaning of entries will be described as follows:

- library_location: The `iTunes Media` folder path
- rhythmbox_playlist: The rhythmbox playlist xml path
- iTunes_db: `iTunes Music Library.xml` path
- skip_list: playlists which you do not want to convert to your Rhythmbox
- source_itunes_music_folder: do not modify it
'''
# Configurations
library_location = '~/Music/Music/Media'

rhythmbox_playlist = '~/.local/share/rhythmbox/playlists.xml'

itunes_db = '~/Music/Music/Library.xml'

skip_list = [
    '资料库',
    '音乐',
    '已下载',
    '播客',
    'Library',
    'Downloaded',
    'Favorites',
    'Music',
    'Movies',
    'TV Shows',
    'Podcasts',
    'Audiobooks',
    'Languages',
    'Special',
    # 'Dislike',
    # 'Not Classified',
    # 'Not Favorate',
    # 'Not Rated',
    'Pomodoro',
    'Voice Memos',
]

MATCH_JUDGEMENT_THRESHOLD = 6
MATCH_SELECTIONS = 10
quote_safe_char = "!$'()*+,-/=@_~"

source_itunes_music_folder = None
# End of configuration

__author__ = "Charles Xu"
__github__ = "https://github.com/the0demiurge"
__girlfriend__ = "pang"
try:
    from urllib import quote, unquote
except ImportError:
    from urllib.parse import quote, unquote


def itunes_quote(string):
    string = quote(string, safe=quote_safe_char)
    string = string.replace('%26', '&')
    return string


library_location_orig = library_location
if not (library_location.startswith('http://') or library_location.startswith('file://') or library_location.startswith('https://')):
    library_location = 'file://' + \
        itunes_quote(os.path.abspath(os.path.expanduser(library_location)))
    if not library_location.endswith('/'):
        library_location += '/'
rhythmbox_playlist, itunes_db = os.path.abspath(os.path.expanduser(rhythmbox_playlist)), os.path.abspath(os.path.expanduser(itunes_db))


def xml2dict(xml_path):
    xml_obj = ElementTree.parse(xml_path)
    root = xml_obj.getroot()

    def traverse(root):
        children = list(root)
        return {root.tag: {'children': [traverse(child) for child in children], 'attrib': root.attrib, 'text': root.text}}

    return traverse(root)


def parse_itunesdb(xml_path):
    xml_obj = ElementTree.parse(xml_path)
    root = list(xml_obj.getroot())[0]

    def traverse(root):
        children = list(root)

        handlers = {
            'dict': lambda root, children: {traverse(key): traverse(value) for key, value in zip(children[::2], children[1::2])} if len(children) % 2 == 0 else ValueError('dict must be paired'),
            'key': lambda root, children: root.text,
            'string': lambda root, children: root.text,
            'date': lambda root, children: root.text,
            'data': lambda root, children: root.text,
            'integer': lambda root, children: root.text,
            'true': lambda root, children: True,
            'false': lambda root, children: False,
            'array': lambda root, children: [traverse(child) for child in children],
        }

        if root.tag in handlers:
            return handlers[root.tag](root, children)
        else:
            raise KeyError(root.tag)

    return traverse(root)


def playlist2rbxml(playlists, destination):
    '''
    Arguments:
        playlists [{'name': str, 'item': [str]}]
    '''
    head = '\n\n'
    playlist_head = '\n'
    song = '{}\n'
    playlist_tail = '\n'
    tail = '\n\n'
    with open(destination, 'w') as f:
        f.write(head)
        for playlist in playlists:
            f.write(playlist_head.format(playlist['name']))
            for track in playlist['item']:
                f.write(song.format(track))
            f.write(playlist_tail)
        f.write(tail)


def get_all_files_path(path):
    result = list()
    path = os.path.abspath(os.path.expanduser(path))
    for agent in os.walk(path):
        for file_name in agent[2]:
            if not file_name.startswith('.'):
                result.append(os.path.join(agent[0], file_name))
    return result


def edit_distance(src, target):
    dp = [[0 for i in range(len(target) + 1)] for j in range(len(src) + 1)]
    for i in range(1, len(src) + 1):
        dp[i][0] = i
    for j in range(1, len(target) + 1):
        dp[0][j] = j
    for i in range(1, len(src) + 1):
        for j in range(1, len(target) + 1):
            dp[i][j] = min(1 + dp[i - 1][j], 1 + dp[i][j - 1], dp[i - 1][j - 1] + (0 if src[i - 1] == target[j - 1] else 1))
    return dp[-1][-1]


def edit_distance_wrapper(data):
    distance = 0
    for src, target in zip_longest(data[0].split('/'), data[1].split('/')):
        if src == target:
            continue
        elif None in (src, target):
            if src is not None:
                distance += len(src)
            elif target is not None:
                distance += len(target)
        else:
            distance += edit_distance(src, target)
    return distance


pool = multiprocessing.Pool()
music_match_buffer = dict()


def music_match(path, music_match_library):
    if path in music_match_buffer:
        return music_match_buffer[path]
    distances = pool.map(edit_distance_wrapper, zip((path,) * len(music_match_library), music_match_library))
    zipper = tuple(zip(distances, music_match_library))
    distance, matched = min(zipper, key=lambda x: x[0])
    if distance >= MATCH_JUDGEMENT_THRESHOLD:
        feature = r'\d*-*[a-zA-Z]*/*\.*'
        candidates = heapq.nsmallest(MATCH_SELECTIONS, zipper, key=lambda x: x[0])
        filtered = [i for i in candidates if ''.join(re.findall(feature, path)) == ''.join(re.findall(feature, i[1]))]
        if len(filtered) == 1:
            distance, matched = filtered[0]
        else:
            distance, matched = min(candidates, key=lambda x: edit_distance(re.findall(feature, path), re.findall(feature, x[1])))

    print('Dist:', distance, 'Matched: "{}" -- "{}"'.format(path.split('/')[-1], matched.split('/')[-1]))
    music_match_buffer[path] = matched
    return matched


def convert_music_path(music_location, source_itunes_music_folder, library_location, music_match_library):
    data = music_location.replace(source_itunes_music_folder, library_location, 1).split('://', 1)
    if len(data) == 1:
        music_file_path = unquote(data[0])
        head = ''
    else:
        music_file_path = unquote(data[1])
        head = data[0] + '://'
    if not os.path.exists(music_file_path):
        matched_music_file_path = music_match(music_file_path, music_match_library)
    else:
        matched_music_file_path = music_file_path
    if music_file_path in music_match_library:
        music_match_library.remove(music_file_path)
    return head + itunes_quote(matched_music_file_path)


def get_playlist(itunes_dict, skip_list=skip_list, convert_function=convert_music_path, **convert_args):
    tracks, playlists = itunes_dict['Tracks'], itunes_dict['Playlists']
    return [{'name': playlist['Name'], 'item': [convert_function(tracks[track['Track ID']]['Location'], **convert_args) for track in playlist['Playlist Items']]} for playlist in playlists if playlist['Name'] not in skip_list and 'Playlist Items' in playlist]


def main():
    music_match_library = get_all_files_path(library_location_orig)
    rhythmbox_pid = list(map(eval, os.popen("ps -A|grep rhythmbox|awk '{print $1}'").readlines()))
    if rhythmbox_pid:
        if '-f' in sys.argv or '--force' in sys.argv:
            prompt = 'y'
        else:
            prompt = input('rhyhtmbox process found: {}, kill it?[Y/n]:'.format(rhythmbox_pid)).lower()
        if prompt == 'n':
            exit()
        elif prompt == 'y' or prompt == '':
            for pid in rhythmbox_pid:
                os.kill(pid, 2)
                print('rhythmbox quitted')
        else:
            print('You should input "y" or "n"')
            exit(1)

    itunes_dict = parse_itunesdb(itunes_db)
    global source_itunes_music_folder
    if not source_itunes_music_folder:
        source_itunes_music_folder = itunes_dict['Music Folder']
    playlists = get_playlist(
        itunes_dict,
        skip_list=skip_list,
        convert_function=convert_music_path,
        source_itunes_music_folder=source_itunes_music_folder,
        library_location=library_location,
        music_match_library=music_match_library
    )
    for playlist in playlists:
        print(len(playlist['item']), playlist['name'], sep='\t')
    playlist2rbxml(playlists, rhythmbox_playlist)
    print(len(playlists), 'playlists imported')


if __name__ == '__main__':
    if 'help' in sys.argv or '-h' in sys.argv or '--help' in sys.argv or not os.path.isfile(itunes_db):
        print(help_message)
    else:
        main()


2019.06.12 更新:同步脚本

我写了一个简单的在mac和其他电脑之间同步音乐文件的脚本,可以自动检测操作系统来判断是传入还是传出。只要把该脚本放到u盘里,每次插入u盘运行该脚本,便能够自动同步音乐文件。当然如果你想使用的话必须要修改里面的两个路径。
同步脚本:
#!/usr/bin/env fish
switch (uname)
    case Linux
        rsync  -rzhu --delete-before --exclude Podcasts --exclude Downloads --info=progress2 --progress /run/media/charles/Charles/iTunes ~/Music/
    case Darwin
        rsync  -rzhu --delete-before --exclude Podcasts --exclude Downloads --info=progress2 --progress ~/Music/iTunes /Volumes/Charles
end
iTunes播放列表转换为rhythmbox脚本:
#!/usr/bin/env python3
import os
import sys
from xml.etree import ElementTree

help_message = '''Usage: Close Rhythmbox, then type: itunes2rhythmbox

If you are to move or sync your whole iTunes library to Rhythmbox, This script
will help you to convert iTunes playlist to Rhythmbox style playlist, and
import the playlist to Rhythmbox directly.

Before using this script, you should have:
- an iTunes folder copied directly from your Windows/Mac computer(with the
  iTunes files already consolidated: in iTunes, click File -> Library ->
  Organize Library -> Consolidate files -> OK), or just the `iTunes Media`
  Folder located at `iTunes/`
- a copy of `iTunes Music Library.xml`, which usually located at `iTunes/`

You should modify this script configurations at start of the script, and the
meaning of entries will be described as follows:

- target_location: The `iTunes Media` folder path
- rhythmbox_playlist: The rhythmbox playlist xml path
- iTunes_db: `iTunes Music Library.xml` path
- skip_list: playlists which you do not want to convert to your Rhythmbox
- source_itunes_music_folder: do not modify it
'''
# Configurations
target_location = '~/Music/iTunes/iTunes Media/'

rhythmbox_playlist = '~/.local/share/rhythmbox/playlists.xml'

itunes_db = '~/Music/iTunes/iTunes Library.xml'

skip_list = [
    '资料库',
    '音乐',
    '已下载',
    '播客',
    'Library',
    'Downloaded',
    'Favorates',
    'Music',
    'Movies',
    'TV Shows',
    'Podcasts',
    'Audiobooks',
    'Languages',
    'Special',
    'Dislike',
    'Not Classified',
    'Not Favorate',
    'Not Rated',
    'Pomodoro',
    'Voice Memos',
]

quote_safe_char = "!$'()*+,-/=@_~"

source_itunes_music_folder = None
# End of configuration

__author__ = "Charles Xu"
__github__ = "https://github.com/the0demiurge"
__girlfriend__ = "pang"
try:
    from urllib import quote, unquote
except ImportError:
    from urllib.parse import quote, unquote


def itunes_quote(string):
    string = quote(string, safe=quote_safe_char)
    string = string.replace('%26', '&')
    return string


if not (target_location.startswith('http://') or target_location.startswith('file://') or target_location.startswith('https://')):
    target_location = 'file://' + \
        itunes_quote(os.path.abspath(os.path.expanduser(target_location)))
    if not target_location.endswith('/'):
        target_location += '/'
rhythmbox_playlist, itunes_db = os.path.abspath(os.path.expanduser(rhythmbox_playlist)), os.path.abspath(os.path.expanduser(itunes_db))


def xml2dict(xml_path):
    xml_obj = ElementTree.parse(xml_path)
    root = xml_obj.getroot()

    def traverse(root):
        children = list(root)
        return {root.tag: {'children': [traverse(child) for child in children], 'attrib': root.attrib, 'text': root.text}}

    return traverse(root)


def parse_itunesdb(xml_path):
    xml_obj = ElementTree.parse(xml_path)
    root = list(xml_obj.getroot())[0]

    def traverse(root):
        children = list(root)

        handlers = {
            'dict': lambda root, children: {traverse(key): traverse(value) for key, value in zip(children[::2], children[1::2])} if len(children) % 2 == 0 else ValueError('dict must be paired'),
            'key': lambda root, children: root.text,
            'string': lambda root, children: root.text,
            'date': lambda root, children: root.text,
            'data': lambda root, children: root.text,
            'integer': lambda root, children: root.text,
            'true': lambda root, children: True,
            'false': lambda root, children: False,
            'array': lambda root, children: [traverse(child) for child in children],
        }

        if root.tag in handlers:
            return handlers[root.tag](root, children)
        else:
            raise KeyError(root.tag)

    return traverse(root)


def playlist2rbxml(playlists, destination):
    '''
    Arguments:
        playlists [{'name': str, 'item': [str]}]
    '''
    head = '\n\n'
    playlist_head = '\n'
    song = '{}\n'
    playlist_tail = '\n'
    tail = '\n\n'
    with open(destination, 'w') as f:
        f.write(head)
        for playlist in playlists:
            f.write(playlist_head.format(playlist['name']))
            for track in playlist['item']:
                f.write(song.format(track))
            f.write(playlist_tail)
        f.write(tail)


def convert_music_path(music_location, source_itunes_music_folder, target_location):
    data = music_location.replace(source_itunes_music_folder, target_location, 1).split(':', 1)
    if len(data) == 1:
        return itunes_quote(unquote(data[0]))
    else:
        return ':'.join((data[0], itunes_quote(unquote(data[1]))))


def get_playlist(itunes_dict, skip_list=skip_list, convert_function=convert_music_path, **convert_args):
    tracks, playlists = itunes_dict['Tracks'], itunes_dict['Playlists']
    return [{'name': playlist['Name'], 'item': [convert_function(tracks[track['Track ID']]['Location'], **convert_args) for track in playlist['Playlist Items']]} for playlist in playlists if playlist['Name'] not in skip_list and 'Playlist Items' in playlist]


def main():
    rhythmbox_pid = list(map(eval, os.popen("ps -A|grep rhythmbox|awk '{print $1}'").readlines()))
    if rhythmbox_pid:
        if '-f' in sys.argv or '--force' in sys.argv:
            prompt = 'y'
        else:
            prompt = input('rhyhtmbox process found: {}, kill it?[Y/n]:'.format(rhythmbox_pid)).lower()
        if prompt == 'n':
            exit()
        elif prompt == 'y' or prompt == '':
            for pid in rhythmbox_pid:
                os.kill(pid, 2)
                print('rhythmbox quitted')
        else:
            print('You should input "y" or "n"')
            exit(1)

    itunes_dict = parse_itunesdb(itunes_db)
    global source_itunes_music_folder
    if not source_itunes_music_folder:
        source_itunes_music_folder = itunes_dict['Music Folder']
    playlists = get_playlist(
        itunes_dict,
        skip_list=skip_list,
        convert_function=convert_music_path,
        source_itunes_music_folder=source_itunes_music_folder,
        target_location=target_location
    )
    for playlist in playlists:
        print(len(playlist['item']), playlist['name'], sep='\t')
    playlist2rbxml(playlists, rhythmbox_playlist)
    print(len(playlists), 'playlists imported')


if __name__ == '__main__':
    if 'help' in sys.argv or '-h' in sys.argv or '--help' in sys.argv or not os.path.isfile(itunes_db):
        print(help_message)
    else:
        main()

 已知问题:日语和韩语部分歌名转换失败,不管怎么处理都没办法转换到rhythmbox播放列表里。

2018.10.13 更新:我买了MacBook

买了MacBook之后发现,谷歌音乐同步原来是支持对 iTunes播放列表识别的,似乎 Linux 和 Windows 都没有这种支持;现在安卓上的音乐同步变得简单了起来,只要有 iTunes 和 Google play music,任何可以上网的设备(包括 Chromebook和安卓手表)都可以同步音乐库,在全设备有着相同的音乐体验。


以下是原文:

多平台同步:



  • 使用 Winodws 或 Mac 的 iTunes 作为中心服务器
  • 使用 isyncr 从 Windows
    同步到安卓设备,安卓设备使用任何可以识别到 m3u 格式的软件(Google play music, poweramp)听歌。注:poweramp 对这些音乐支持的最好。
  • iTunes 可以直接同步到 iPod/iPhone
  • 使用 rsync 将整个 iTunes 文件夹同步到 Linux,使用 itunes2rhythmbox
rsync 同步 bat 脚本:itunes-sync.bat
X:
cd X:\music
rsync -e ssh -rzhu --delete-before --info=progress2 --progress iTunes user@ip_address:~/Music
pause
将 iTunes 库转换为 Rhythmbox 的库脚本:itunes2rhythmbox
https://github.com/the0demiurge/CharlesScripts/blob/master/charles/bin/itunes2rhythmbox

这样,在某些地方听到好听的歌就下载整个专辑到 iTunes,然后同步到所有设备即可;一切操作都在中心服务器中运行,而一旦中心服务器崩溃,Linux 里面还有一份 iTunes文件夹的完整拷贝,迁移到 Mac 或 Windows 都十分容易。

下载歌曲:


先 Google,看看能不能找到无损;不能的话就用国产音乐软件下载非无损也行,最好每次下载一整个专辑,再把这整个专辑打好评分。

无损的音乐使用WinMount/DAEMON Tools Lite/foobar2000/dBpoweramp Music Converter,挂载为虚拟光盘让 iTunes 扫描,或直接转换成 Apple Loosless,补充完整标签即可。

不足之处:


  1. 依赖 iTunes,而 iTunes 不支持 Linux
  2. Winodws 和 Mac 互相同步音乐库的时候会遇到音乐找不到的问题,写个脚本处理一下 iTunes 数据库应该就可以,不过我不打算同时使用 Windows 和 Mac 的 iTunes。

解决方案:
  1. 弄个 Windows 或 Mac 当中心音乐服务器/虚拟机
  2. 如果同时 Win 和 Mac 都有了,那就只选其中一个听歌呗

评论

此博客中的热门博文

Flash被淘汰后打开swf文件的最佳方法

[SOLVED] Supermicro cannot connect to VGA video port or iKVM

MacBook日文键盘四种输入模式输入法切换(同样适用于其他布局的键盘)