Firstly note, this is not about ASM, rather much the opposite, to describe the historical considerations to take into account to make Oracle run well on filesystems such as UFS and VXFS. Some folks not into using ASM yet might find this useful.

This is largely a description of how to work with a sysadmin and/or storage admin to get some filesystem storage that you'll be happy with Oracle performance on.

The first thing is to identify the native OS page size, and consider this as the smallest useful i/o unit, and choose your Oracle instance default blocksize to match native OS page size. And ensure the filesystem blocksize matches those other 2 choices. On Solaris platform, for example, UFS and VXFS default to allowing 1k frag sizes, which is not useful for Oracle.

Start with understanding your hard drives, what is the make and model? Dig up a spec sheet, and identify characteristics such as the capacity, RPMs, the number of cylinders, the size of a cylinder (e.g. disk capacity / number of cylinders). The cylinder is the largest unit of storage that can be read without performing a seek. This is about the largest useful i/o unit, and a fair guide of how you might like to size your extents on large tables, temp and undo segments.

when filesystems are built take into account the smallest and largest practical i/o units. fs blocksize = os page size = oracle default blocksize. ufs maxcontig ~ cylinder ~ large table extent size.

Corresponding filesystem creation options for an 8k blocksize and 1M maxcontig would be like...
newfs -o bsize=8192,fragsize=8192,maxcontig=1048576

Now actually, common large disks now have cylinders larger than 1MB, but some i/o drivers have 1MB max design limit, so you may have some artificial limits that don't match optimal geometry.