OMAP4 pandaboard上CPU hotplug进程迁移与负载均衡测试

尝试把pandaboard images升级为 11.12 Linaro release，SMP热插拔测试了一下。第一组测试评估在高负载情况下热插拔CPU进程迁移的情况，希望是拔出和插入CPU1后，各进程仍然活跃，测试case如下：

int main(int argc, char *argv[])
{
	struct timeval last_tv, cur_tv;
	int pid;
	char p_proc[50], on = '1', off = '0';
	int fd = open("/sys/devices/system/cpu/cpu1/online", O_RDWR);
	if (fd < 0)
		exit (0);

	sprintf(p_proc, "/proc/%d", getpid());

	gettimeofday (&last_tv , NULL);

	/* fork to 16 processes with high cpu usage */
	pid = fork();
	if (pid > 0) {
		printf("before unplugging cpu1\n");
		sleep(5);
		/* unplug cpu1 */
		write(fd, &off, 1);
		printf("after unplugging cpu1, we want to see all processes are still alive\n");
		sleep(5);
		/* re-plug cpu1 */
		printf("after re-plugging cpu1\n");
		write(fd, &on, 1);
		sleep(5);
	} else {
		/* lauch multi-processes with 100% cpu usage */
		fork();		fork();		fork();		fork();
		while(1) {
			gettimeofday (&cur_tv , NULL);
			if (cur_tv.tv_sec - last_tv.tv_sec == 4) {
				printf("pid: %d\n", getpid());
				last_tv = cur_tv;
			}
			/* if parent process exit, sub-process exit */
			if (access(p_proc, 0) < 0)
				_exit(0);
		}
	}
}

Fork出16个CPU利用率100%的进程，在热插拔CPU的情况下，这些16个进程保持活跃，证明热插拔的情况下，进程可迁移：

root@android:/ # smp-unplug-process-alive
before unplugging cpu1
pid: 582 pid: 583 pid: 576 pid: 579 pid: 574 pid: 587 pid: 580 pid: 586
pid: 581 pid: 588 pid: 575 pid: 573 pid: 577 pid: 584 pid: 578 pid: 585
[   68.295593] CPU1: shutdown
after unplugging cpu1, we want to see all processes are still alive
pid: 587 pid: 581 pid: 578 pid: 576 pid: 582 pid: 585 pid: 577 pid: 574
pid: 573 pid: 580 pid: 586 pid: 579 pid: 583 pid: 575 pid: 584 pid: 588
after re-plugging cpu1
[   73.520935] CPU1: Booted secondary processor
[   73.528350] Switched to NOHz mode on CPU #1
pid: 588 pid: 581 pid: 582 pid: 576 pid: 586 pid: 585 pid: 579 pid: 584
pid: 587 pid: 573 pid: 578 pid: 580 pid: 577 pid: 583 pid: 575 pid: 574
…

第二组测试评估2个processes并行计算下，CPU hotplug后对时间的影响，以彻底验证拔掉CPU1后只有CPU0在运算，重新插入后2个开始一起运算，测试case如下：

int main(int argc, char *argv[])
{
    volatile int i;
    int pid = fork();
    
    if (pid > 0) {
        for (i=0;i<90000000;i++);
        wait(0);
    } else {
        for (i=0;i<90000000;i++);
    }
}

在不unplug cpu1的情况下：

root@android:/ # time /system/bin/smp-time
    0m0.73s real     0m1.42s user     0m0.03s system
root@android:/ # time /system/bin/smp-time
    0m0.72s real     0m1.41s user     0m0.01s system
root@android:/ # time /system/bin/smp-time
    0m0.72s real     0m1.43s user     0m0.00s system

实际运行时间0.73秒，在2个CPU上累积耗费了1.42秒, 各自贡献1/2力量。
现在我们拔出cpu1，再运行：

echo 0 > /sys/devices/system/cpu/cpu1/online                                                
[  234.565521] CPU1: shutdown
root@android:/ #  time /system/bin/smp-time
    0m1.45s real     0m1.38s user     0m0.06s system
root@android:/ #  time /system/bin/smp-time
    0m1.45s real     0m1.44s user     0m0.00s system
root@android:/ #  time /system/bin/smp-time
    0m1.45s real     0m1.45s user     0m0.00s system

实际运行耗时1.45秒，等于user耗时，因为现在只有1个CPU0顺序执行2个进程。
我们把CPU1插入，再执行：

echo 1 > /sys/devices/system/cpu/cpu1/online
[  364.515075] CPU1: Booted secondary processor
root@android:/ # time /system/bin/smp-time
    0m0.72s real     0m1.42s user     0m0.00s system
root@android:/ # time /system/bin/smp-time
    0m0.72s real     0m1.43s user     0m0.00s system

真实时间又变成user时间1/2，证明CPU1活跃，开始与CPU0并行运算。
结论是Linux kernel有较好的CPU热插拔支持，在热插拔情况下，进程能够顺利迁移与再次负载均衡。