HPUNIX 11.11 rman备份，每秒不到20M~

Yong Huang

Good finding. So RMAN is doing the right thing. But OS is not. Maybe platform-specific Oracle manual has some recommendation?
http://docs.oracle.com/cd/E11882_01/relnotes.112/e23561/toc.htm
http://docs.oracle.com/cd/E11882_01/install.112/e24336/toc.htm

Look for discussion on not breaking a big I/O request into small chunks, if there's any such discussion. Not specifically about RMAN. Can you check with iostat to see if the I/O chunks are also 8k when you run a query instead of using RMAN to backup?

What's the setting of disk_async_io and filesystemio_options?

You're right about v$backup_async_io. V$backup_sync_io is about Sync I/O only.

zergduan

Yong Huang 发表于 2012-8-14 23:07
Good finding. So RMAN is doing the right thing. But OS is not. Maybe platform-specific Oracle manual ...

感谢您的回复~ 我今天测试一下

zergduan

Hi Huang

1。关于AIO,CIO:
这个数据库的参数配置如下
disk_async_io = true
filesystemio_options = async

但是由于所有的数据文件放在VXFS V4的文件系统上，是没法使用AIO的~ MOS note: HP-UX: Asynchronous i/o [ID 139272.1]
甚至因为VXFS版本太低，也没法使用CIO (要求VXFS 5.01以上)~ MOS note: How to use Concurrent I/O on HP-UX and improve throughput on an Oracle single-instance database [ID 1231869.1]http://mylinux.5d6d.net/thread-574-1-1.html


所以这台机器也没有配置AIO; 没有创建/dev/async设备~ 所以上面的两个参数应该是无效的~

2。关于Can you check with iostat to see if the I/O chunks are also 8k when you run a query instead of using RMAN to backup?

这个数据库的有关参数如下:
db_block_size=8k
db_file_multiblock_read_count=8

我在session级别修改db_file_multiblock_read_count=128，然后用select /*+ full(b) */ count(*) from b的方式发起一个每个IO 1MB的查询，同时观察iostat输出； 正如您说的和RMAN一样，每个IO都变成了8KB...

zergduan

我在9i 的release notes /installation guide中，都没有找到关于IO breaking的信息~

zergduan

马上要下班了，我发现http://h20000.www2.hp.com/bc/doc ... 19408/c01919408.pdf 提到了一些vxfs参数，好像和文件系统上发生的IO有关

zergduan

上面的文章中提到


On HP-UX 11i v2 and earlier, cp(1) reads and writes data using 64 KB logical I/O. Using dd, data
can be read and written using 256 KB I/Os or larger. The large I/O size will cause dd to engage the
Discovered Direct I/O feature of HP OnlineJFS and the data will be transferred using large direct I/O.
There are several advantages of using dd(1) over cp(1):
 Transfer can be done using a larger I/O transfer size
 Buffer cache is bypassed, thus leaving other more important data in the cache.
 Data is written synchronously, instead of asynchronously, avoiding large buildup of dirty buffers
which can potentially cause large I/O queues or process that call sync()/fsync() to temporarily
hang

这似乎解释了为什么我的cp和dd可以用比较大的IO完成~

看样子我这个问题应该应该是vxfs中跟Buffered/cached I/O有关的参数问题，明天上班的时候看看哪些参数~

zergduan

上面的文章中:

If data needs to be accessed once without keeping the data in the cache, various options such as using direct I/O, discovered direct I/O, or the VX_SETCACHE ioctl with the VX_NOREUSE option may be used. For example, rather than using cp(1) to copy a large file, try using dd(1) instead using a large block size as follows:
# dd if=srcfile of=destfile bs=1024k
On HP-UX 11i v2 and earlier, cp(1) reads and writes data using 64 KB logical I/O. Using dd, data can be read and written using 256 KB I/Os or larger. The large I/O size will cause dd to engage the Discovered Direct I/O feature of HP OnlineJFS and the data will be transferred using large direct I/O. There are several advantages of using dd(1) over cp(1):
 Transfer can be done using a larger I/O transfer size
 Buffer cache is bypassed, thus leaving other more important data in the cache.
 Data is written synchronously, instead of asynchronously, avoiding large buildup of dirty buffers which can potentially cause large I/O queues or process that call sync()/fsync() to temporarily hang.
这正好解释了为什么我cp和dd可以用比较大的IO来完成，因为那是direct IO~

看样子我这个问题应该和vxfs的Buffered/cached I/O的相关参数有关

Yong Huang

> 我在session级别修改db_file_multiblock_read_count=128，然后用select /*+ full(b) */ count(*) from b
> 的方式发起一个每个IO 1MB的查询，同时观察iostat输出； 正如您说的和RMAN一样，每个IO都变成了8KB...

So now we can say the problem is not with RMAN because even a simple query is doing 8k at a time. See if HP or Veritas (Symantec) has any recommendation, or open an SR with Oracle.

zergduan

Yong Huang 发表于 2012-8-16 03:11
> 我在session级别修改db_file_multiblock_read_count=128，然后用select /*+ full(b) */ count(*) from b
...

谢谢
上面的文章提到了If data needs to be accessed once without keeping the data in the cache, various options such as using direct I/O, discovered direct I/O, or the VX_SETCACHE ioctl with the VX_NOREUSE option may be used. For example, rather than using cp(1) to copy a large file, try using dd(1) instead using a large block size as follows:
# dd if=srcfile of=destfile bs=1024k
On HP-UX 11i v2 and earlier, cp(1) reads and writes data using 64 KB logical I/O. Using dd, data can be read and written using 256 KB I/Os or larger. The large I/O size will cause dd to engage the Discovered Direct I/O feature of HP OnlineJFS and the data will be transferred using large direct I/O. There are several advantages of using dd(1) over cp(1):
 Transfer can be done using a larger I/O transfer size
 Buffer cache is bypassed, thus leaving other more important data in the cache.
 Data is written synchronously, instead of asynchronously, avoiding large buildup of dirty buffers which can potentially cause large I/O queues or process that call sync()/fsync() to temporarily hang.

anlinew

Block size
The block size (bsize) is the smallest amount of space that can be allocated to a file extent. Most
applications will perform better with an 8 KB block size. Extent allocations are easier and file systems
with an 8 KB block size are less likely to be impacted by fragmentation since each extent would have
a minimum size of 8 KB. However, using an 8 KB block size could waste space, as a file with 1 byte
of data would take up 8 KB of disk space.
The default block size for a file system scales depending on the size of the file system when the file
system is created. The fstyp(1M) command can be used verify the block size of a specific file system
using the f_frsize field. The f_bsize can be ignored for VxFS file systems as the value will always be
8192.
# fstyp -v /dev/vg00/lvol3
vxfs
version: 6
f_bsize: 8192
f_frsize: 8192