Skip to content

基于 electron 开发在线课堂

此文不讲整体项目思路,只是对该项目开发的一些踩坑总结,备忘用。

相关链接

音视频流

腾讯云

注:

  1. 在线课堂 electron 应用可以基于实时互动课堂(Electron)开发,本质是实时音视频trtc+即时通讯im的组合。
  2. 虽然 electron 本质是打包 web 应用,但由于腾讯云 SDK 实现问题,TRTC Web SDK 并不能用在 electron 应用里,可能会有兼容问题。

electron

记录点

异步获取系统设备

js
export const getDeviceList = async (type: MediaType): Promise<[Device | null, Device[]]> => {
  if (!deviceInfo) {
    const devices = await navigator.mediaDevices.enumerateDevices();
    const deviceMap: DeviceMap = {
      audiooutput: 'speaker',
      audioinput: 'microphone',
      videoinput: 'camera'
    };
    const curDeviceInfo: DeviceInfo = {};

    devices.forEach(({ deviceId, kind, label }) => {
      if (!curDeviceInfo[deviceMap[kind]]) {
        curDeviceInfo[deviceMap[kind]] = [];
      }
      curDeviceInfo[deviceMap[kind]]!.push({
        deviceId,
        kind,
        label,
        isCurrent: deviceId === 'default'
      });
    });
    Object.values(curDeviceInfo).forEach((item: Device[]) => {
      if (item.length === 1) {
        item[0].isCurrent = true;
      }
    });
    deviceInfo = curDeviceInfo;
  }
  const list = deviceInfo[type]!;
  if (!list || list.length === 0) {
    return [null, []];
  }
  const curDevice = list!.filter((device) => device.isCurrent)[0];
  return [curDevice, list];
};

选择对应扬声器

js
const selectSpeackerDevice =
  (current: HTMLMediaElement): ((id: string) => void) =>
  (id: string) => {
    const currentId = counterId;
    current
      .setSinkId(id)
      .then(() => {
        current.currentTime = 0;
        if (currentId !== counterId) {
          current.pause();
          return;
        }
        current.play();
      })
      .catch((e) => {
        message.error(`获取扬声器失败!`);
        console.log(e);
      });
  };

清除 media 流及动画

js
export const clearMediaAndAnimate = () => {
  if (mediaStream) {
    mediaStream.getTracks()[0].stop();
  }
  cancelAnimationFrame(animate);
  counterId += 1;
};

选择对应麦克风并绘制音量图谱

js
const setCanvas = (current: HTMLCanvasElement) => {
  const { width, height } = current;
  const canvasCtx = current.getContext('2d')!;
  canvasCtx.clearRect(0, 0, width, height);
  canvasCtx.fillStyle = '#fff';
  canvasCtx.fillRect(0, 0, width, height);

  return (info: number) => {
    canvasCtx.clearRect(0, 0, width, height);
    canvasCtx.fillRect(0, 0, info, height);

    for (let i = 0; i < info; i += 1) {
      canvasCtx.beginPath();
      canvasCtx.lineWidth = 2;
      canvasCtx.strokeStyle = i % 2 ? `#437BFF` : '#fff';
      canvasCtx.moveTo(i, 0);
      canvasCtx.lineTo(i, height);
      canvasCtx.stroke();
    }
  };
};

const selectMicrophoneDevice = (current: HTMLCanvasElement): ((id: string) => void) => (id: string) => {
  const currentId = counterId;
  const constraints = {
    audio: { deviceId: { exact: id } }
  };
  const drawCanvas = setCanvas(current);
  const audioCtx = new AudioContext();
  const analyser = audioCtx.createAnalyser(); // 频率及时间域分析器
  analyser.fftSize = 256;
  let source;

  navigator.mediaDevices
    .getUserMedia(constraints)
    .then((stream) => {
      mediaStream = stream;
      if (currentId !== counterId) {
        clearMediaAndAnimate();
        return;
      }
      source = audioCtx.createMediaStreamSource(stream); // 创建源
      source.connect(analyser);
      const dataArray = new Uint8Array(analyser.frequencyBinCount);
      const draw = () => {
        animate = requestAnimationFrame(draw);
        analyser.getByteFrequencyData(dataArray);
        drawCanvas(dataArray[0]);
      };
      draw();
    })
    .catch((e) => {
      message.error(`获取麦克风失败!`);
      console.log(e);
      clearMediaAndAnimate();
    });
};

选择对应摄像头

js
const selectCameraDevice =
  (current: HTMLVideoElement): ((id: string) => void) =>
  (id: string) => {
    const currentId = counterId;
    const constraints = {
      video: { deviceId: { exact: id } },
    };
    navigator.mediaDevices
      .getUserMedia(constraints)
      .then((stream) => {
        mediaStream = stream;
        if (currentId !== counterId) {
          clearMediaAndAnimate();
          return;
        }
        current.srcObject = stream;
      })
      .catch((e) => {
        message.error(`获取视频流失败!`);
        console.log(e);
        clearMediaAndAnimate();
      });
  };

其中通过 counterId 保留当前最新流,异步丢弃之前旧流。

electron-builder.json

json
{
  "productName": "electron客户端",
  "appId": "cn.electron.korey",
  "copyright": "Copyright © 2021 korey",
  "asar": true,
  "compression": "maximum", //若用 store,则打包速度加快,但打包体积变大
  "nsis": {
    "oneClick": false, //取消一键安装
    "allowElevation": true,
    "allowToChangeInstallationDirectory": true,
    "installerIcon": "./resources/icons/icon.ico", // 256*256
    "uninstallerIcon": "./resources/icons/icon.ico",
    "installerHeaderIcon": "./resources/icons/icon.ico",
    "createDesktopShortcut": true,
    "createStartMenuShortcut": true,
    "shortcutName": "electron客户端"
  },
  "files": ["dist/", "node_modules/", "app.html", "main.prod.js", "main.prod.js.map", "package.json"],
  "dmg": {
    "contents": [
      {
        "x": 130,
        "y": 220
      },
      {
        "x": 410,
        "y": 220,
        "type": "link",
        "path": "/Applications"
      }
    ]
  },
  "win": {
    "extraFiles": [
      {
        "from": "node_modules/trtc-electron-sdk/build/Release", //将 .node 文件复制过去,没有这个程序将报错
        "to": "."
      }
    ],
    "target": {
      "target": "nsis",
      "arch": "x64"
    },
    "icon": "./resources/icons/icon.ico" //256*256 ico格式,未配 icon 则 win 打包报错
  },
  "mac": {
    "category": "zhibojiaoyu.app",
    "extraFiles": [
      {
        "from": "node_modules/trtc-electron-sdk/build/Release", //同上
        "to": "./Frameworks"
      }
    ]
  },
  "linux": {
    "target": ["deb", "rpm", "AppImage"],
    "category": "Development"
  },
  "directories": {
    "buildResources": "resources",
    "output": "release"
  },
  "publish": {
    "provider": "generic",
    "channel": "latest",
    "url": "/img/electron客户端-1.0.0.dmg",
    "private": false
  },
  "electronDownload": {
    "mirror": "https://npm.taobao.org/mirrors/electron/"
  }
}

腾讯云相关

  • 直播群 AVChatRoom(需求大于 6000 人)不支持历史消息存储。群组系统对比
  • startScreenCapture 开启屏幕推流后,可通过 setSubStreamMixVolume 设置麦克风和屏幕里音源大小比例。win 上需异步调用 startSystemAudioLoopback 才能采集到屏幕里音源。其中默认摄像头为主流,屏幕为辅流。
  • IM 群组已存在,除直播群需要同时调 joinGroup 以外,其他类型再次 createGroup 会直接进入该群组。
  • trtc enterRoom roomId 取值范围 1~4294967295
  • trtc getScreenCaptureSourcesmac os big sur 版本返回的 screenList.thumbBGRA 里的 width*height*4 不等于 buffer.length 导致程序报错,等待腾讯云修复。
  • 窗口置顶:setAlwaysOnTop(true, 'pop-up-menu'), 一定要有 pop-up-menu 参数,因为在 win 上无此参数时分享全屏屏幕时,拖动置顶窗口会意外置底。

sdk node 支持

webpack 需配置解析腾讯云 sdk .noderules

json
{
  "test": /\.node$/,
  "loader": "native-ext-loader",
  "options": {
    "emit": false,
    "rewritePath": process.env.NODE_ENV === "production" ? "./" : "node_modules/trtc-electron-sdk/build/Release/"
  }
}

BrowserWindow 配置里需加上 webPreferences: {nodeIntegration: true}

其他

  • 使用 setBounds 代替 setSize,因为 setSizewin 上多次调用会失效。
  • CSS 中指定 -webkit-app-region: drag 来告诉 Electron 哪些区域可拖拽。